INTEGRATION [PR#2471 > development/2.14] MongoDbPodRecovering alert firing once per reporting pod instead of once per recovering member - #2481
Open
bert-e wants to merge 5 commits into
Conversation
…e per recovering member mongodb_rs_members_state has one series per (reporting pod, member): every pod's exporter reports the state of every replica-set member, so a single recovering member matched once per reporting pod and the description named the reporter instead of the recovering member. With one member recovering in a 3-member shard, the alert fired three times, naming the healthy PRIMARY and SECONDARY as recovering. In this commit we aggregate the metric by member_idx so exactly one alert fires per recovering member, and rebuild the pod label from the first DNS segment of member_idx so the description names the recovering member's pod. This also keeps the alert firing when the recovering member's own exporter cannot be scraped, since its peers still report it. unit tests now reflect realistic series (every pod reporting every member) covering both the nominal case and an unscrapeable recovering member; the previous test used a single synthetic series and could not catch the fan-out. Issue: ZENKO-5332
Issue: ZENKO-5332
The 5 'Promote artifacts' tests fail on development/2.13 even though everything is pinned (action-gh-release@v2.5.0, act 0.2.75): the catthehacker act image (pulled with forcePull) now ships node 24 in its toolcache, which changed how unmatched mocked requests fail. action-gh-release@v2.5.0 never calls getReleaseByTag: it scans listReleases (findTagFromReleases) before creating the release. The second mocked listReleases reply contained the draft release, so the pre-creation scan found the target tag and took the update-existing path, issuing an updateRelease call no mock matches. This used to pass by accident: the unmatched request surfaced as a fast 404 which the action recovered from by creating the release; on node 24 it now hangs until an undici headers timeout and fails the step. Make the second listReleases reply draft-free so the action takes the create path, with every request matched by a mock. scality/action-artifacts@v4 (floating tag) is now a node24 action, which act 0.2.75 refuses to run: bump act to 0.2.84 as on development/2.14, and adapt the step-result parsing to the timing suffix newer act appends to step status lines. Issue: ZENKO-5317
Issue: ZENKO-5250 (cherry picked from commit cc68501)
…-member' into w/2.14/improvement/ZENKO-5332-mongodb-pod-recovering-alert-per-member
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request has been created automatically.
It is linked to its parent pull request #2471.
Do not edit this pull request directly.
If you need to amend/cancel the changeset on branch
w/2.14/improvement/ZENKO-5332-mongodb-pod-recovering-alert-per-member, please follow thisprocedure:
git fetch git checkout w/2.14/improvement/ZENKO-5332-mongodb-pod-recovering-alert-per-member # <amend or cancel the changeset by _adding_ new commits> git push origin w/2.14/improvement/ZENKO-5332-mongodb-pod-recovering-alert-per-memberPlease always comment pull request #2471 instead of this one.