Skip to content

docs: updated isolation.level description#19628

Open
avaamsel wants to merge 2 commits into
apache:masterfrom
avaamsel:master
Open

docs: updated isolation.level description#19628
avaamsel wants to merge 2 commits into
apache:masterfrom
avaamsel:master

Conversation

@avaamsel

Copy link
Copy Markdown

Updated the description for isolation.level in Kafka ingestion docs.

This PR has:

  • [x ] been self-reviewed.

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reviewed the code for correctness, edge cases, concurrency, and integration risks; no issues found.

Reviewed 2 of 2 changed files.


This is an automated review by Codex GPT-5.5

@317brian 317brian left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nit

The `isolation.level` property in `consumerProperties` determines how Druid reads messages written transactionally.
With `read_committed`, which is the default in Druid, only committed transactions are read.
If you use older versions of Kafka without transaction support, or you want to read even aborted transactions, set `isolation.level` to `read_uncommitted`.
The `isolation.level` property determines how Druid handles transactional Kafka messages. Although standard Kafka consumers default to `read_uncommitted`, Druid's ingestion engine defaults `read_committed`. This ensures that only finalized data is indexed and aborted transactions are ignored. If you need to use legacy Kafka brokers or don’t want Druid to consume only committed transactions, explicitly set `isolation.level` to `read_uncommitted`. Note that using `read_uncommitted` removes Druid's offset gap check, which requires the message source to ensure the message offsets are continuous.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The `isolation.level` property determines how Druid handles transactional Kafka messages. Although standard Kafka consumers default to `read_uncommitted`, Druid's ingestion engine defaults `read_committed`. This ensures that only finalized data is indexed and aborted transactions are ignored. If you need to use legacy Kafka brokers or don’t want Druid to consume only committed transactions, explicitly set `isolation.level` to `read_uncommitted`. Note that using `read_uncommitted` removes Druid's offset gap check, which requires the message source to ensure the message offsets are continuous.
The `isolation.level` property determines how Druid handles transactional Kafka messages. Although standard Kafka consumers default to `read_uncommitted`, Druid's ingestion engine defaults to `read_committed`. This ensures that only finalized data is indexed and aborted transactions are ignored. If you need to use legacy Kafka brokers or don’t want Druid to consume only committed transactions, explicitly set `isolation.level` to `read_uncommitted`. Note that using `read_uncommitted` removes Druid's offset gap check, which requires the message source to ensure the message offsets are continuous.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants