Skip to content

#108 Normalize misspelled HTTP headers#2006

Open
HarshDevelops wants to merge 1 commit into
apache:mainfrom
HarshDevelops:fix/108-spellchecked-http-headers
Open

#108 Normalize misspelled HTTP headers#2006
HarshDevelops wants to merge 1 commit into
apache:mainfrom
HarshDevelops:fix/108-spellchecked-http-headers

Conversation

@HarshDevelops

@HarshDevelops HarshDevelops commented Jul 19, 2026

Copy link
Copy Markdown

Thank you for contributing to Apache StormCrawler.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

For all changes

  • Is there a issue associated with this PR? Is it referenced in the commit message?

  • Does your PR title start with #XXXX where XXXX is the issue number you are trying to resolve?

  • Has your PR been rebased against the latest commit within the target branch (typically main)?

  • Is your initial contribution a single, squashed commit?

  • Is the code properly formatted with mvn git-code-format:format-code -Dgcf.globPattern="**/*" -Dskip.format.code=false?

For code changes

  • Have you ensured that the full suite of tests is executed via mvn clean verify?
  • Have you written or updated unit tests to verify your changes?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE file, including the main LICENSE file?
  • If applicable, have you updated the NOTICE file, including the main NOTICE file?

Note

Please ensure that once the PR is submitted, you check GitHub Actions for build issues and submit an update to your PR as soon as possible.

Closes #108

Summary:

  • Normalize HTTP header metadata when a known header name is misspelled or its hyphens are omitted.
  • Leave non-HTTP metadata unchanged.
  • Add regression coverage for both cases.

Testing:

  • mvn -pl core -Dtest=MetadataTest test
  • mvn -pl core test
  • mvn git-code-format:format-code -Dgcf.globPattern="**/*" -Dskip.format.code=false
  • mvn --no-transfer-progress clean verify reached stormcrawler-opensearch, then failed locally because Docker/Testcontainers was unavailable; skipped tests then caused the module JaCoCo threshold to fail.

@jnioche

jnioche commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

I like the purpose but would rather have that outside of Metadata itself. We have comparable utility classes in o.a.s.util.*, this would be the right place for it I think.
Metadata is not just about HTTP Headers, I'd rather keep it slim. Does that make sense @HarshDevelops ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Normalise HTTP headers

2 participants