Skip to content

CAMEL-23530: Move DataWeave parser to camel-dataweave component and add auto-detection in DataSonnet - #25363

Merged
davsclaus merged 6 commits into
mainfrom
feature/CAMEL-23530-dataweave-component
Aug 6, 2026
Merged

CAMEL-23530: Move DataWeave parser to camel-dataweave component and add auto-detection in DataSonnet#25363
davsclaus merged 6 commits into
mainfrom
feature/CAMEL-23530-dataweave-component

Conversation

@davsclaus

@davsclaus davsclaus commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Moves the DataWeave AST parser/lexer/converter from camel-jbang-core into a new components/camel-dataweave component (org.apache.camel.component.dataweave) for broader reuse. The module is a tiny pure-JDK JAR with zero runtime dependencies.
  • Adds camel-dataweave as a dependency of camel-datasonnet so that .dwl files and inline %dw expressions are automatically transpiled to DataSonnet at route initialization time — zero configuration needed.
  • Auto-detection works two ways: by file extension (.dwl) and by content header (%dw).
  • Updates datasonnet-language.adoc with a new "DataWeave Support" section documenting auto-detection, examples in Java/XML/YAML, and the CLI transpiler.
  • Fixes DataWeave auto-detection: moves detection to createExpression(String, Object[]) so .dwl extension check runs before the parent class loads the resource (previously the extension check was dead code). Also logs at WARN when constructs cannot be auto-converted.

Test plan

  • camel-dataweave: 89 tests pass (moved tests + existing)
  • camel-datasonnet: 67 tests pass (65 existing + 2 new DataWeave auto-detect tests)
  • Verify .dwl resource auto-detection via resource:classpath:transform.dwl
  • Verify inline %dw expression auto-detection
  • Verify no regression in existing DataSonnet functionality

🤖 Generated with Claude Code

@davsclaus davsclaus changed the title CAMEL-23530: Move DataWeave parser to camel-dataweave component CAMEL-23530: Move DataWeave parser to camel-dataweave component and add auto-detection in DataSonnet Aug 5, 2026
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

davsclaus and others added 4 commits August 5, 2026 20:38
…euse

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
Add camel-dataweave dependency to camel-datasonnet so .dwl files and
inline %dw expressions are automatically transpiled to DataSonnet at
route initialization time with zero configuration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…structs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
@davsclaus
davsclaus force-pushed the feature/CAMEL-23530-dataweave-component branch from 3fe4062 to 3d7ebc1 Compare August 5, 2026 18:38
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

🧪 CI tested the following changed modules:

  • bom/camel-bom
  • catalog/camel-allcomponents
  • catalog/camel-catalog
  • components
  • components/camel-datasonnet
  • components/camel-dataweave
  • docs
  • dsl/camel-jbang/camel-jbang-core
  • parent

🔬 Scalpel shadow comparison — Scalpel: 15 tested, 25 compile-only — current: 10 all tested

Maveniverse Scalpel detected 40 affected modules (current approach: 10).

⚠️ Modules only in Scalpel (30)
  • apache-camel
  • camel-allcomponents
  • camel-bom
  • camel-catalog
  • camel-catalog-console
  • camel-catalog-lucene
  • camel-catalog-maven
  • camel-catalog-suggest
  • camel-componentdsl
  • camel-csimple-maven-plugin
  • camel-endpointdsl
  • camel-endpointdsl-support
  • camel-itest
  • camel-jbang-core
  • camel-jbang-it
  • camel-jbang-main
  • camel-jbang-plugin-edit
  • camel-jbang-plugin-generate
  • camel-jbang-plugin-kubernetes
  • camel-jbang-plugin-test
  • camel-kamelet-main
  • camel-launcher
  • camel-report-maven-plugin
  • camel-route-parser
  • camel-yaml-dsl
  • camel-yaml-dsl-deserializers
  • camel-yaml-dsl-maven-plugin
  • coverage
  • docs
  • dummy-component

Changed managed dependencies: org.apache.camel:camel-dataweave

Skip-tests mode would test 15 modules (7 direct + 8 downstream), skip tests for 25 (generated code, meta-modules)

Modules Scalpel would test (15)
  • camel-allcomponents
  • camel-bom
  • camel-catalog
  • camel-datasonnet
  • camel-dataweave
  • camel-jbang-core
  • camel-jbang-mcp
  • camel-jbang-plugin-mcp
  • camel-jbang-plugin-route-parser
  • camel-jbang-plugin-tui
  • camel-jbang-plugin-validate
  • camel-launcher-container
  • camel-yaml-dsl-validator
  • camel-yaml-dsl-validator-maven-plugin
  • docs
Modules with tests skipped (25)
  • apache-camel
  • camel-catalog-console
  • camel-catalog-lucene
  • camel-catalog-maven
  • camel-catalog-suggest
  • camel-componentdsl
  • camel-csimple-maven-plugin
  • camel-endpointdsl
  • camel-endpointdsl-support
  • camel-itest
  • camel-jbang-it
  • camel-jbang-main
  • camel-jbang-plugin-edit
  • camel-jbang-plugin-generate
  • camel-jbang-plugin-kubernetes
  • camel-jbang-plugin-test
  • camel-kamelet-main
  • camel-launcher
  • camel-report-maven-plugin
  • camel-route-parser
  • camel-yaml-dsl
  • camel-yaml-dsl-deserializers
  • camel-yaml-dsl-maven-plugin
  • coverage
  • dummy-component

ℹ️ Shadow mode — Scalpel observes but does not affect test execution. Learn more

⚠️ Some tests are disabled on GitHub Actions (@DisabledIfSystemProperty(named = "ci.env.name")) and require manual verification:

  • dsl/camel-jbang/camel-jbang-core: 1 test(s) disabled on GitHub Actions
  • components: 103 test(s) disabled on GitHub Actions

💡 Manual integration tests recommended:

You modified dsl/camel-jbang/camel-jbang-core. The related integration tests in dsl/camel-jbang/camel-jbang-it are excluded from CI. Consider running them manually:

mvn verify -f dsl/camel-jbang/camel-jbang-it -Djbang-it-test
All tested modules (40 modules)
  • Camel :: All Components Sync point
  • Camel :: Assembly
  • Camel :: BOM
  • Camel :: Catalog :: CSimple Maven Plugin (deprecated)
  • Camel :: Catalog :: Camel Catalog
  • Camel :: Catalog :: Camel Report Maven Plugin
  • Camel :: Catalog :: Camel Route Parser
  • Camel :: Catalog :: Console
  • Camel :: Catalog :: Dummy Component
  • Camel :: Catalog :: Lucene (deprecated)
  • Camel :: Catalog :: Maven
  • Camel :: Catalog :: Suggest
  • Camel :: Component DSL
  • Camel :: Coverage
  • Camel :: DataSonnet
  • Camel :: DataWeave
  • Camel :: Docs
  • Camel :: Endpoint DSL
  • Camel :: Endpoint DSL :: Support
  • Camel :: Integration Tests
  • Camel :: JBang :: Core
  • Camel :: JBang :: Integration tests
  • Camel :: JBang :: MCP
  • Camel :: JBang :: Main
  • Camel :: JBang :: Plugin :: Edit
  • Camel :: JBang :: Plugin :: Generate
  • Camel :: JBang :: Plugin :: Kubernetes
  • Camel :: JBang :: Plugin :: MCP
  • Camel :: JBang :: Plugin :: Route Parser
  • Camel :: JBang :: Plugin :: TUI
  • Camel :: JBang :: Plugin :: Testing
  • Camel :: JBang :: Plugin :: Validate
  • Camel :: Kamelet Main
  • Camel :: Launcher
  • Camel :: Launcher :: Container
  • Camel :: YAML DSL
  • Camel :: YAML DSL :: Deserializers
  • Camel :: YAML DSL :: Maven Plugins
  • Camel :: YAML DSL :: Validator
  • Camel :: YAML DSL :: Validator Maven Plugin

⚙️ View full build and test results

@gnodet gnodet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean extraction of the DataWeave transpiler into a reusable camel-dataweave module. The auto-detection flow in DataSonnet is well-designed — createExpression(String, Object[]) properly handles .dwl resource detection and %dw content detection before delegating, and the removal of the redundant loadResource() from the 3-arg method is correct since SingleInputTypedLanguageSupport.createExpression(String, Object[]) already handles resource loading. The module is correctly integrated into the BOM, parent POM, catalog, and component reactor. CI passes.

Two minor notes:

  • The generated dataweave.json title uses "Dataweave" (lowercase 'w') while the codebase consistently uses "DataWeave" — worth aligning in the source metadata.
  • Some useful class-level Javadoc and inline comments explaining non-obvious test edge cases were dropped during the move from camel-jbang-core (e.g., tokenizer edge-case coverage rationale in DataWeaveLexerTest).

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

Claude Code on behalf of @gnodet

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
@davsclaus

Copy link
Copy Markdown
Contributor Author

Both review suggestions addressed in 2b47741:

  1. Title casing: Added <title>DataWeave</title> to pom.xml properties so the generated metadata uses the correct casing instead of "Dataweave".

  2. Javadoc and inline comments: Restored the class-level Javadoc on DataWeaveLexerTest and DataWeaveParserTest, and restored inline comments explaining edge-case rationale across all three test files.

@davsclaus davsclaus added this to the 4.22.0 milestone Aug 6, 2026
@davsclaus davsclaus self-assigned this Aug 6, 2026
@davsclaus davsclaus added the enhancement New feature or request label Aug 6, 2026
@davsclaus
davsclaus merged commit f56b04b into main Aug 6, 2026
6 checks passed
@davsclaus
davsclaus deleted the feature/CAMEL-23530-dataweave-component branch August 6, 2026 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants