debezium: output start_ts in the source block of DML messages (#5903)

[ekexium]

What problem does this PR solve?

Issue Number: close #5904

Cherry-pick of #5903 (merge commit 596decb) to release-8.5.

What is changed and how it works?

Add a changefeed-level option (default false) for Debezium JSON output: the sink URI parameter debezium-include-start-ts=true, or in the changefeed config:

[sink.debezium]
include-start-ts = true

When enabled, Debezium JSON DML value messages carry source.start_ts (the start TSO of the transaction) next to commit_ts; the JSON source schema declares the field. DDL/checkpoint messages and key messages are unchanged. The decoder reads source.start_ts when present and falls back to commit_ts for pre-feature or invalid values.

Adaptations relative to master (conflict resolution), all due to features that do not exist on release-8.5:

• pkg/sink/codec/debezium/avro.go / avro_test.go dropped: release-8.5 has no Debezium-Avro codec, so the Avro exclusion changes are not needed here.
• api/v2/changefeed_toml_test.go dropped: TOML output for changefeed query (api/v2: support TOML output for changefeed query via content negotiation #5357) is not on release-8.5; v2 DebeziumConfig keeps json-only tags per branch convention.
• writeSourceSchema keeps the release-8.5 signature style (no schemaName parameter; no EnableTiDBExtension schema block on this branch) and only adds the includeStartTs parameter.
• pkg/sink/codec/common/config.go validation: only the debezium-include-start-ts protocol gate is added; the schema-registry rejection referencing debezium-avro and config.ProtocolDebeziumAvro do not exist on this branch. The added test uses canal-json as the non-debezium protocol.

Check List

Tests

• Unit test (pkg/sink/codec/debezium, pkg/sink/codec/common, api/v2, pkg/config all pass on this branch)
• Canonical test plan, coverage matrix, and execution records: https://pingcap.feishu.cn/docx/TwTKdETqSoGsbJx4gqBcjZ4Qnwh

Questions

Will it cause performance regression or break compatibility?

No. With the default config (option off) output is byte-identical to before. With the option on, the source block gains one uint64 field and the schema declares it consistently. The decoder falls back to commit_ts when start_ts is absent or invalid.

Do you need to update user documentation, design documentation or monitoring documentation?

Yes — same follow-up documentation PR as #5903 (Debezium protocol and changefeed configuration docs).

Release note

Add an option to include the transaction start TSO in Debezium JSON messages. Configure it with the URI parameter `debezium-include-start-ts` or `[sink.debezium] include-start-ts`; when enabled, messages carry `source.start_ts`.

[ti-chi-bot]

This cherry pick PR is for a release branch and has not yet been approved by triage owners.
Adding the do-not-merge/cherry-pick-not-approved label.

To merge this cherry pick:

1. It must be LGTMed and approved by the reviewers firstly.
2. For pull requests to TiDB-x branches, it must have no failed tests.
3. AFTER it has lgtm and approved labels, please wait for the cherry-pick merging approval from triage owners.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a4d662b9-9c2c-40d0-80c8-00648fe8a28c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[ti-chi-bot]

@ekexium: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-error-log-review	5dc09c5	link	true	/test pull-error-log-review

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[Copilot]

Pull request overview

Adds an opt-in Debezium JSON enhancement (release-8.5 cherry-pick of #5903) to include the transaction start_ts in DML message payload.source, along with schema declaration and decoder support, while keeping backward compatibility via fallback behavior.

Changes:

• Add debezium-include-start-ts (URI) / [sink.debezium] include-start-ts (config) to control emitting source.start_ts for Debezium JSON DML messages.
• Update Debezium JSON encoder/schema generation to optionally emit/declare start_ts for DML only (DDL/checkpoint unchanged).
• Update Debezium decoder to read source.start_ts when present and fall back to commit_ts for older or invalid messages; add unit tests and API v2 model/config plumbing.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
pkg/sink/codec/debezium/decoder.go	Decode source.start_ts with fallback to commit_ts for backward compatibility.
pkg/sink/codec/debezium/debezium_test.go	Add encode/schema/round-trip and fallback tests for start_ts.
pkg/sink/codec/debezium/codec.go	Conditionally emit and declare start_ts in Debezium JSON DML source block/schema.
pkg/sink/codec/debezium/codec_test.go	Assert DDL/checkpoint messages do not declare/contain start_ts.
pkg/sink/codec/common/config.go	Add and apply debezium-include-start-ts sink URI/config handling and validation gate.
pkg/sink/codec/common/config_test.go	Test URI/config precedence and protocol validation for the new option.
pkg/config/sink.go	Add Debezium config field IncludeStartTs and a default constant for output-old-value.
pkg/config/replica_config.go	Use the Debezium output-old-value default constant in defaults.
api/v2/model.go	Plumb Debezium include_start_ts through API v2 and fix defaulting for output_old_value.
api/v2/model_test.go	Add coverage for Debezium config conversion/defaulting including include_start_ts.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[ekexium]

@wk989898 @lidezhu PTAL

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2026-08-19 10:38:10.495766017 +0000 UTC m=+66125.666860134: ☑️ agreed by wk989898.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: wk989898
Once this PR has been reviewed and has the lgtm label, please assign lidezhu for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS [wk989898]
• pkg/config/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment