[hashicorp_vault] feat: populate and parse user_agent from audit request User-Agent header#20122
Open
lyoung-confluent wants to merge 2 commits into
Open
[hashicorp_vault] feat: populate and parse user_agent from audit request User-Agent header#20122lyoung-confluent wants to merge 2 commits into
lyoung-confluent wants to merge 2 commits into
Conversation
…er-Agent header WHAT: In the hashicorp_vault `audit` data stream ingest pipeline, add a painless script that reads the User-Agent header from `hashicorp_vault.audit.request.headers.user-agent` (a flattened field whose values Vault emits as arrays) and populates `user_agent.original`. The header value is unwrapped from its list form (falling back to a scalar) and skipped when it starts with `hmac-`, since Vault HMACs header values when configured to do so and the HMAC digest is not a meaningful user agent. A subsequent `user_agent` processor parses `user_agent.original` into the standard ECS subfields, matching the pattern used across the rest of the repo. The parsed ECS fields are declared in `fields/ecs.yml`. WHY: Surfacing the request User-Agent as structured ECS `user_agent.*` fields makes Vault audit events consistent with other integrations and enables filtering/aggregation by client without parsing raw headers. The `test-faked-all-fields` pipeline test is extended (no new document added): one existing request event now carries a real User-Agent to exercise parsing, and one existing response event carries an HMAC'd User-Agent to verify the skip path leaves `user_agent` unset. Verified locally with `elastic-package test pipeline` (all pass) and `--generate` (no diff). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
ReviewersBuildkite won't run for external contributors automatically; you need to add a comment:
NOTE: https://github.com/elastic/integrations/blob/main/.buildkite/pull-requests.json contains all those details. |
lyoung-confluent
changed the title
Author
|
It's worth noting that by default Vault does NOT log |
Replace the painless script that populated `user_agent.original` with a `set` processor using `copy_from` on the first User-Agent header element (`hashicorp_vault.audit.request.headers.user-agent.0`) and `ignore_empty_value`. The HMAC-skip condition (Vault can be configured to HMAC header values, and the digest is not a meaningful user agent) moves into the processor's `if`. This is more declarative and idiomatic than the script while producing identical output; verified with `elastic-package test pipeline` (all pass). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed commit message
WHAT:
In the
hashicorp_vaultauditdata stream ingest pipeline, add a painless script that reads theUser-Agentheader fromhashicorp_vault.audit.request.headersfield and populatesuser_agent.original. The header value is unwrapped from its list form and skipped when it starts withhmac-, since Vault can be configured to HMAC header values and the digest is not a meaningful user agent. A subsequentuser_agentprocessor then parsesuser_agent.originalinto the standard ECS subfields, matching the pattern used across the rest of the repo. The parsed ECS fields are declared infields/ecs.yml.WHY:
Surfacing the request User-Agent as structured ECS
user_agent.*fields makes Vault audit events consistent with other integrations and enables filtering/aggregation by client without parsing raw headers.Checklist
changelog.ymlfile.How to test this PR locally
The
test-faked-all-fieldspipeline test was extended (no new document added):User-Agent(Go-http-client/2.0) to exercise parsing intouser_agent.*.User-Agentto verify the skip path leavesuser_agentunset.Verified locally with
elastic-package test pipeline(all pass) and--generate(no diff).Related issues
None