Skip to content

Add steps for local data ingestion in docs#7790

Closed
sumairq wants to merge 31 commits intomozilla:masterfrom
sumairq:ingestion-docs
Closed

Add steps for local data ingestion in docs#7790
sumairq wants to merge 31 commits intomozilla:masterfrom
sumairq:ingestion-docs

Conversation

@sumairq
Copy link
Copy Markdown
Contributor

@sumairq sumairq commented Sep 1, 2023

This PR updates the docs on steps to ingest data locally.

It is a solution for https://bugzilla.mozilla.org/show_bug.cgi?id=1850748

  • It includes steps to help contributors set up a local data ingestion environment.

Copy link
Copy Markdown
Collaborator

@gmierz gmierz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work @sumairq :) just a few additional things we'll need to add to make it clearer for others.

Comment thread docs/pulseload.md Outdated
Comment thread docs/pulseload.md Outdated
Comment thread docs/pulseload.md Outdated
Comment thread docs/pulseload.md Outdated
Comment thread docs/pulseload.md Outdated
Copy link
Copy Markdown
Collaborator

@Archaeopteryx Archaeopteryx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for extending the documentation.

Comment thread docs/pulseload.md Outdated
Comment thread docs/pulseload.md Outdated
Comment thread docs/pulseload.md Outdated
Comment thread docs/pulseload.md Outdated
@gmierz
Copy link
Copy Markdown
Collaborator

gmierz commented Sep 2, 2023

@sumairq looking at the other documentation we have, and I'm wondering if it would make more sense to have this information on this page (maybe in the ingesting pushes section): https://treeherder.readthedocs.io/installation.html

What do you think?

@sumairq
Copy link
Copy Markdown
Contributor Author

sumairq commented Sep 2, 2023

@sumairq looking at the other documentation we have, and I'm wondering if it would make more sense to have this information on this page (maybe in the ingesting pushes section): https://treeherder.readthedocs.io/installation.html

What do you think?

Maybe we can keep the information under the Ingesting pushes section the way it is and include a link to the detailed setup in the Loading pulse data section?

Copy link
Copy Markdown
Collaborator

@Archaeopteryx Archaeopteryx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you.

Comment thread docs/pulseload.md Outdated

---

##### Run the following in separate window while running above to do ingestion
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @sumairq, sorry for the delayed review! In the image below, I noticed that this line is showing up as a new heading in the navigation list/bar on the right-hand side. Could you reformat this line to be in bold instead so it doesn't show up there?

image

@gmierz
Copy link
Copy Markdown
Collaborator

gmierz commented Sep 27, 2023

a link to the detailed setup in the Loading pulse data section?

I'm not sure about this, why do you think the Loading pulse data section would be better? While navigating locally, I found that it's a bit harder to find it here versus the installation page.

beatrice-acasandrei and others added 13 commits September 28, 2023 14:06
Bug 1827920 - also get the `updatePinnedJob` method in the local scope to fix sporadic push health issue

The import was being used instead of the prop tied to the store resulting in the `updatePinnedJob` prop sometimes having the value of `null` when it is attempted to be called later inside FailureSummaryTab.
* Update Node.js to v20.6.1

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Bug 1838836 - Add TaskclusterMetadata to the alert summary API endpoint
* Base the query on JobLog table

* Filter from the database directly

* Refactor the Python code

* Add distinct clause

* Force ordering of the entire payload

* Add a comment for summarygroup result direct annotation
* Bug 1850785 - Allow for Marionette failures to be filed as single tracking bugs. r=aryx

* allow mitten to work with cf jobs, fix bug with -cf at end of jobname.
Previously, select all checkboxes in the tests tab would always stay on the same position on screen due to a missing positive relative.
@sumairq sumairq requested a review from a team as a code owner September 29, 2023 15:46
@sumairq sumairq requested a review from gmierz September 29, 2023 15:48
@gmierz
Copy link
Copy Markdown
Collaborator

gmierz commented Oct 4, 2023

@sumairq would you be able to fix up this PR to only show the files modified by your PR? It looks like there are merge conflicts as well that need to be resolved. I think a forced push from an up-to-date branch could help.

@beatrice-acasandrei
Copy link
Copy Markdown
Collaborator

beatrice-acasandrei commented May 6, 2026

Due to the recent inactivity we can close this PR, this issue will be wrapped up in #9477.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants