From 6f88ab7e594f446b81a7b563f20847d507af02df Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 7 Jul 2026 15:24:51 +0800 Subject: [PATCH 01/11] Create empty translation PR From 30871ca02e8bb18e1ecba44327856e9884fc87fe Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Tue, 7 Jul 2026 07:26:48 +0000 Subject: [PATCH 02/11] Auto-sync: Update English docs from Chinese PR Synced from: https://github.com/pingcap/docs-cn/pull/21723 Target PR: https://github.com/pingcap/docs/pull/23211 AI Provider: azure Co-authored-by: github-actions[bot] --- TOC.md | 1 + ticdc/ticdc-changefeed-config.md | 3 +- ticdc/ticdc-table-routing.md | 240 +++++++++++++++++++++++++++++++ 3 files changed, 242 insertions(+), 2 deletions(-) create mode 100644 ticdc/ticdc-table-routing.md diff --git a/TOC.md b/TOC.md index b00beca582739..1e5eb2d15b018 100644 --- a/TOC.md +++ b/TOC.md @@ -66,6 +66,7 @@ - [Replicate Data to Storage Services](/ticdc/ticdc-sink-to-cloud-storage.md) - [Manage Changefeeds](/ticdc/ticdc-manage-changefeed.md) - [Log Filter](/ticdc/ticdc-filter.md) + - [Table Routing](/ticdc/ticdc-table-routing.md) - [DDL Replication](/ticdc/ticdc-ddl.md) - [Bidirectional Replication](/ticdc/ticdc-bidirectional-replication.md) - Monitor and Alert diff --git a/ticdc/ticdc-changefeed-config.md b/ticdc/ticdc-changefeed-config.md index 37afc633077ce..29a9a9d74b593 100644 --- a/ticdc/ticdc-changefeed-config.md +++ b/ticdc/ticdc-changefeed-config.md @@ -182,10 +182,9 @@ For more information, see [Event filter rules](/ticdc/ticdc-filter.md#event-filt #### `dispatchers` -- For the sink of MQ type, you can use dispatchers to configure the event dispatcher. +- In versions earlier than v8.5.7, `dispatchers` takes effect only when the downstream is an MQ sink and is used to configure event dispatchers. Starting from v8.5.7, for the [new TiCDC architecture](/ticdc/ticdc-architecture.md), `dispatchers` can also be used to configure table routing, mapping upstream tables to specified downstream database names or table names. For more information, see [TiCDC table routing](/ticdc/ticdc-table-routing.md). - Starting from v6.1.0, TiDB supports two types of event dispatchers: partition and topic. - The matching syntax of matcher is the same as the filter rule syntax. -- This configuration item only takes effect if the downstream is MQ. - When the downstream MQ is Pulsar, if the routing rule for `partition` is not specified as any of `ts`, `index-value`, `table`, or `default`, each Pulsar message will be routed using the string you set as the key. For example, if you specify the routing rule for a matcher as the string `code`, then all Pulsar messages that match that matcher will be routed with `code` as the key. #### `column-selectors` New in v7.5.0 diff --git a/ticdc/ticdc-table-routing.md b/ticdc/ticdc-table-routing.md new file mode 100644 index 0000000000000..ec12d79187958 --- /dev/null +++ b/ticdc/ticdc-table-routing.md @@ -0,0 +1,240 @@ +--- +title: TiCDC Table Routing +summary: Learn about table routing configuration in the TiCDC new architecture, including how to rewrite downstream database and table names using target-schema and target-table, how to handle DDL rewriting and routing conflicts, and how to troubleshoot common issues. +--- + +# TiCDC Table Routing Introduced in v8.5.7 + +TiCDC table routing allows you to map upstream tables to specified downstream database names or table names through changefeed configuration. This feature applies only to the [TiCDC new architecture](/ticdc/ticdc-architecture.md) and is not supported by the [TiCDC old architecture](/ticdc/ticdc-classic-architecture.md). + +Table routing changes only the database and table names that TiCDC outputs downstream. It does not change row data, column names, table schema, table filter rules, Topic dispatch rules, Partition dispatch rules, or column selector rules. + +## Usage scenarios + +Table routing is suitable for the following scenarios: + +- Synchronizing `sales.orders` to `archive.sales_orders`, or to a table that follows other downstream naming conventions. +- Synchronizing multiple source databases to the same target database while keeping target table names unique to distinguish the source, for example, synchronizing `tenant_001.orders` to `tenant_mirror.tenant_001_orders`. +- Building migration, disaster recovery, archiving, or shadow Changefeeds to avoid writing to downstream objects with the same names as upstream ones. +- Exposing stable database and table names to MQ consumer applications or storage services. + +> **Note:** +> +> Table routing supports only one-to-one mapping from an upstream table to a downstream target table. It does not support merging multiple upstream tables into the same downstream table. +> Table routing does not support splitting one upstream table into multiple downstream tables, nor transforming row data content. + +## Configure table routing + +Before configuration, enable the TiCDC new architecture. For details, see [`newarch`](/ticdc/ticdc-server-config.md#newarch-从-v854-release1-版本开始引入). + +The following example routes `sales.orders` to `archive.sales_orders`: + +```toml +[sink] +[[sink.dispatchers]] +matcher = ["sales.orders"] +target-schema = "archive" +target-table = "{schema}_{table}" +``` + +After the Changefeed starts, the DML and DDL events of `sales.orders` are written to the downstream `archive.sales_orders`. + +> **Note:** +> +> Different tables in the same upstream database can be routed to different target databases. This kind of configuration applies only to DML and table-level DDL. +> +> - For database-level DDL such as `CREATE DATABASE`, `DROP DATABASE`, and `ALTER DATABASE`, TiCDC must be able to determine a unique target database based on the routing rules. Otherwise, synchronization of that DDL fails. +> - If you want to route different tables in the same upstream database to different target databases, you need to create the downstream target databases in advance and avoid executing upstream database-level DDL that requires automatic synchronization. + +## Configuration fields + +The table routing feature uses [`sink.dispatchers`](/ticdc/ticdc-changefeed-config.md#dispatchers) as the configuration entry. + +| Field | Description | +| :--- | :--- | +| `matcher` | Matches upstream databases and tables. The syntax is the same as [table filter syntax](/table-filter.md#表库过滤语法), including wildcard matches such as `sales.*` and exclusion matches such as `!sales.tmp_*`. | +| `target-schema` | Specifies the downstream database name. If this field is not set, TiCDC keeps the upstream database name unchanged. | +| `target-table` | Specifies the downstream table name. If this field is not set, TiCDC keeps the upstream table name unchanged. | + +The matching behavior is as follows: + +- Only `sink.dispatchers` rules with `target-schema` or `target-table` set participate in table routing. +- If a table matches multiple table routing rules, only the first matching rule in `sink.dispatchers` takes effect. +- `matcher` always matches upstream database and table names, not the routed target database and table names. +- The Changefeed configuration item `case-sensitive` affects only whether the `matcher` in table routing is case-sensitive. It does not rewrite the expansion results of `{schema}` and `{table}`. For details, see [`case-sensitive`](/ticdc/ticdc-changefeed-config.md#case-sensitive). + +### Placeholders + +You can use the following placeholders in `target-schema` and `target-table`: + +| Placeholder | Description | +| :--- | :--- | +| `{schema}` | The upstream database name, preserving the actual matched database name case. | +| `{table}` | The upstream table name, preserving the actual matched table name case. | + +The values of `target-schema` and `target-table` can contain only literal text, `{schema}`, and `{table}`. If you use an unknown placeholder such as `{db}`, TiCDC rejects the Changefeed configuration and returns the `CDC:ErrInvalidTableRoutingRule` error. + +Using the source table `sales.orders` as an example: + +| Configuration | Target table | +| :--- | :--- | +| `target-schema = "archive"` | `archive.orders` | +| `target-table = "{table}_bak"` | `sales.orders_bak` | +| `target-schema = "{schema}_mirror"` | `sales_mirror.orders` | +| `target-schema = "archive"` and `target-table = "{schema}_{table}"` | `archive.sales_orders` | + +## Examples + +### Route all tables in one database + +The following configuration routes all tables in the `sales` database to the `archive` database and appends `_bak` to the target table names: + +```toml +[filter] +rules = ["sales.*"] + +[sink] +[[sink.dispatchers]] +matcher = ["sales.*"] +target-schema = "archive" +target-table = "{table}_bak" +``` + +Example routing results: + +- `sales.orders` is routed to `archive.orders_bak`. +- `sales.order_items` is routed to `archive.order_items_bak`. + +### Route multiple databases to the same target database + +When routing multiple databases to the same target database, it is recommended to include `{schema}` in `target-table` to ensure unique target table names. + +```toml +[filter] +rules = ["sales.*", "crm.*", "finance.*"] + +[sink] +[[sink.dispatchers]] +matcher = ["sales.*", "crm.*", "finance.*"] +target-schema = "archive" +target-table = "{schema}_{table}" +``` + +Example routing results: + +- `sales.orders` is routed to `archive.sales_orders`. +- `crm.orders` is routed to `archive.crm_orders`. +- `finance.orders` is routed to `archive.finance_orders`. + +> **Note:** +> +> This configuration applies only to database merging, not table merging. Table routing does not support merging same-named tables from multiple different databases into the same downstream table. + +### Use table routing together with Kafka Sink Topic and Partition dispatchers + +The same dispatcher rule can include both table routing fields and existing dispatch fields: + +```toml +[filter] +rules = ["sales.orders"] + +[sink] +[[sink.dispatchers]] +matcher = ["sales.orders"] +topic = "order-events" +partition = "index-value" +target-schema = "public" +target-table = "orders" +``` + +In the preceding example, table routing changes the database and table names exposed in downstream data to `public.orders`. + +Table routing does not change the dispatch results of Topic or Partition. The `topic` and `partition` dispatchers still perform matching and dispatch calculation based on the upstream table `sales.orders`. + +## Output behavior + +| Sink | Behavior | +| :--- | :--- | +| MySQL Sink | DDL and DML statements are written to the routed target database and table. When the Redo feature is enabled, executing `redo apply` replays events to the routed target table. | +| Kafka Sink and Pulsar Sink | The values of the `payload` field in the protocol and the `query` field in DDL events use the routed target database and table names; the values of the `schema` and `table` fields in the encoding protocol also use the routed target database and table names. | +| Cloud Storage Sink | TiCDC generates the corresponding storage paths, schema files, table definition files, and data files based on the routed target database and table names. | + +## DDL behavior + +After table routing is enabled, TiCDC rewrites DDL statements so that the database and table names in structured DDL metadata remain consistent with those in the SQL text. + +For example, if the following rule is configured: + +```toml +[[sink.dispatchers]] +matcher = ["sales.*"] +target-schema = "archive" +target-table = "{table}_routed" +``` + +TiCDC rewrites the following upstream DDL: + +```sql +RENAME TABLE `sales`.`temp_table` TO `sales`.`renamed_table`; +``` + +into the following downstream DDL: + +```sql +RENAME TABLE `archive`.`temp_table_routed` TO `archive`.`renamed_table_routed`; +``` + +If a DDL statement contains table references and those table references match table routing rules, TiCDC also rewrites the referenced table names. For example, table references in `CREATE VIEW` statements and foreign key references in `ALTER TABLE` statements can be rewritten. + +For database-level DDL such as `CREATE DATABASE`, `DROP DATABASE`, and `ALTER DATABASE ... CHARACTER SET/COLLATE`, if the database name matches table routing rules, TiCDC rewrites the database name. **If the same upstream database matches multiple table routing rules and these rules resolve to different target database names, TiCDC cannot determine a unique target database for that database-level DDL, and the Changefeed returns a table routing error.** + +When creating or updating a Changefeed, TiCDC checks for target table conflicts based on the tables currently within the synchronization scope. During runtime, TiCDC updates the conflict detection state when synchronizing DDL such as `CREATE TABLE`, `RENAME TABLE`, `DROP TABLE`, and `DROP DATABASE`. Whether a database-level DDL can determine a unique target database is evaluated when synchronizing the corresponding DDL. +## Route conflict detection + +A route conflict occurs when two different upstream tables are routed to the same downstream `(schema, table)` within a single Changefeed. TiCDC does not support merging multiple upstream tables into the same downstream table. + +For example, the following configuration might cause a conflict: + +```toml +[filter] +rules = ["sales.*", "crm.*"] + +[sink] +[[sink.dispatchers]] +matcher = ["sales.*"] +target-schema = "archive" +target-table = "{table}" + +[[sink.dispatchers]] +matcher = ["crm.*"] +target-schema = "archive" +target-table = "{table}" +``` + +If both `sales.orders` and `crm.orders` are within the synchronizing scope, both tables are routed to `archive.orders`. TiCDC rejects the creation or update of the Changefeed and returns the `CDC:ErrTableRouteConflict` error. + +While the Changefeed is running, if a DDL such as `CREATE TABLE` or `RENAME TABLE` causes two upstream tables currently within the synchronizing scope to be routed to the same target table, the Changefeed fails and returns the `CDC:ErrTableRouteConflict` error. + +If an upstream table is dropped or renamed, TiCDC removes the routing relationship between that upstream table and the target table. After that, a new upstream table can continue to use that target table. However, at any given time, the same target table can correspond to only one upstream table being synchronized by the current Changefeed. + +> **Warning:** +> +> Route conflict detection is limited to a single Changefeed. If multiple Changefeeds write to the same downstream system, make sure that the table routing rules of these Changefeeds do not write to the same target object. + +## Troubleshooting + +| Symptom | Possible cause | Solution | +| :--- | :--- | :--- | +| The `CDC:ErrInvalidTableRoutingRule` error is reported when creating or updating a Changefeed. | The `matcher` syntax is invalid, or `target-schema` or `target-table` contains unknown placeholders or mismatched braces. | Check whether `matcher` conforms to the [table filter syntax](/table-filter.md#表库过滤语法), and make sure that `target-schema` and `target-table` use only literal text, `{schema}`, and `{table}`. | +| The MQ Topic name still uses the upstream schema and table name. | Table routing does not change Topic or Partition dispatch rules. | If you need to change the Topic name, configure `topic` separately in `sink.dispatchers`. | +| The `CDC:ErrTableRoutingFailed` error is reported during DDL synchronization. | The DDL cannot be safely rewritten for table routing, or a schema-level DDL has ambiguity in the target schema. | Check the DDL type and routing rules. For schema-level DDL, make sure the same upstream schema can be parsed to only one target schema. | +| The Changefeed fails during runtime and reports the `CDC:ErrTableRouteConflict` error. | After a table is created or renamed, two different upstream tables are routed to the same downstream table. | Adjust the table routing rules or upstream DDL to ensure that, within a single Changefeed, each target table corresponds to only one upstream table being synchronized by the current Changefeed at any given time. | + +## Related documentation + +- [TiCDC Changefeed command-line flags and configuration parameters](/ticdc/ticdc-changefeed-config.md) +- [Changefeed event filter](/ticdc/ticdc-filter.md) +- [Synchronize data to MySQL-compatible databases](/ticdc/ticdc-sink-to-mysql.md) +- [Synchronize data to Kafka](/ticdc/ticdc-sink-to-kafka.md) +- [Synchronize data to Pulsar](/ticdc/ticdc-sink-to-pulsar.md) +- [Synchronize data to storage services](/ticdc/ticdc-sink-to-cloud-storage.md) \ No newline at end of file From f39e19a8df50901f53af71bd5bfd1cd28f1fb8ea Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 7 Jul 2026 15:32:03 +0800 Subject: [PATCH 03/11] Update ticdc/ticdc-table-routing.md --- ticdc/ticdc-table-routing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ticdc/ticdc-table-routing.md b/ticdc/ticdc-table-routing.md index ec12d79187958..464931088c6d6 100644 --- a/ticdc/ticdc-table-routing.md +++ b/ticdc/ticdc-table-routing.md @@ -3,7 +3,7 @@ title: TiCDC Table Routing summary: Learn about table routing configuration in the TiCDC new architecture, including how to rewrite downstream database and table names using target-schema and target-table, how to handle DDL rewriting and routing conflicts, and how to troubleshoot common issues. --- -# TiCDC Table Routing Introduced in v8.5.7 +# TiCDC Table Routing New in v8.5.7 TiCDC table routing allows you to map upstream tables to specified downstream database names or table names through changefeed configuration. This feature applies only to the [TiCDC new architecture](/ticdc/ticdc-architecture.md) and is not supported by the [TiCDC old architecture](/ticdc/ticdc-classic-architecture.md). From 43a83bf29c386cfd88279e75b9030d7dcb263ce7 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 7 Jul 2026 16:00:12 +0800 Subject: [PATCH 04/11] Apply suggestions from code review Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- ticdc/ticdc-changefeed-config.md | 2 +- ticdc/ticdc-table-routing.md | 61 ++++++++++++++++---------------- 2 files changed, 32 insertions(+), 31 deletions(-) diff --git a/ticdc/ticdc-changefeed-config.md b/ticdc/ticdc-changefeed-config.md index 29a9a9d74b593..9d2f510ccfcd1 100644 --- a/ticdc/ticdc-changefeed-config.md +++ b/ticdc/ticdc-changefeed-config.md @@ -182,7 +182,7 @@ For more information, see [Event filter rules](/ticdc/ticdc-filter.md#event-filt #### `dispatchers` -- In versions earlier than v8.5.7, `dispatchers` takes effect only when the downstream is an MQ sink and is used to configure event dispatchers. Starting from v8.5.7, for the [new TiCDC architecture](/ticdc/ticdc-architecture.md), `dispatchers` can also be used to configure table routing, mapping upstream tables to specified downstream database names or table names. For more information, see [TiCDC table routing](/ticdc/ticdc-table-routing.md). +- In versions earlier than v8.5.7, you can use `dispatchers` to configure event dispatchers, and it takes effect only when the downstream is an MQ sink. Starting from v8.5.7, for the [new TiCDC architecture](/ticdc/ticdc-architecture.md), you can also use `dispatchers` to configure table routing, mapping upstream tables to specified downstream database names or table names. For more information, see [TiCDC table routing](/ticdc/ticdc-table-routing.md). - Starting from v6.1.0, TiDB supports two types of event dispatchers: partition and topic. - The matching syntax of matcher is the same as the filter rule syntax. - When the downstream MQ is Pulsar, if the routing rule for `partition` is not specified as any of `ts`, `index-value`, `table`, or `default`, each Pulsar message will be routed using the string you set as the key. For example, if you specify the routing rule for a matcher as the string `code`, then all Pulsar messages that match that matcher will be routed with `code` as the key. diff --git a/ticdc/ticdc-table-routing.md b/ticdc/ticdc-table-routing.md index 464931088c6d6..d7623c62114af 100644 --- a/ticdc/ticdc-table-routing.md +++ b/ticdc/ticdc-table-routing.md @@ -5,16 +5,16 @@ summary: Learn about table routing configuration in the TiCDC new architecture, # TiCDC Table Routing New in v8.5.7 -TiCDC table routing allows you to map upstream tables to specified downstream database names or table names through changefeed configuration. This feature applies only to the [TiCDC new architecture](/ticdc/ticdc-architecture.md) and is not supported by the [TiCDC old architecture](/ticdc/ticdc-classic-architecture.md). +TiCDC table routing allows you to map upstream tables to specified downstream database names or table names through changefeed configuration. This feature applies only to the [TiCDC new architecture](/ticdc/ticdc-architecture.md) and is not supported by the [TiCDC classic architecture](/ticdc/ticdc-classic-architecture.md). -Table routing changes only the database and table names that TiCDC outputs downstream. It does not change row data, column names, table schema, table filter rules, Topic dispatch rules, Partition dispatch rules, or column selector rules. +Table routing changes only the database and table names that TiCDC outputs downstream. It does not change row data, column names, table schema, table filter rules, topic dispatch rules, partition dispatch rules, or column selector rules. ## Usage scenarios Table routing is suitable for the following scenarios: -- Synchronizing `sales.orders` to `archive.sales_orders`, or to a table that follows other downstream naming conventions. -- Synchronizing multiple source databases to the same target database while keeping target table names unique to distinguish the source, for example, synchronizing `tenant_001.orders` to `tenant_mirror.tenant_001_orders`. +- Replicating `sales.orders` to `archive.sales_orders`, or to a table that follows other downstream naming conventions. +- Replicating multiple source databases to the same target database while keeping target table names unique to distinguish the source, for example, replicating `tenant_001.orders` to `tenant_mirror.tenant_001_orders`. - Building migration, disaster recovery, archiving, or shadow Changefeeds to avoid writing to downstream objects with the same names as upstream ones. - Exposing stable database and table names to MQ consumer applications or storage services. @@ -25,7 +25,7 @@ Table routing is suitable for the following scenarios: ## Configure table routing -Before configuration, enable the TiCDC new architecture. For details, see [`newarch`](/ticdc/ticdc-server-config.md#newarch-从-v854-release1-版本开始引入). +Before configuring table routing, enable the TiCDC new architecture. For details, see [`newarch`](/ticdc-server-config.md#newarch-new-in-v854-release1). The following example routes `sales.orders` to `archive.sales_orders`: @@ -37,14 +37,14 @@ target-schema = "archive" target-table = "{schema}_{table}" ``` -After the Changefeed starts, the DML and DDL events of `sales.orders` are written to the downstream `archive.sales_orders`. +After the changefeed starts, TiCDC writes the DML and DDL events of `sales.orders` to the downstream `archive.sales_orders`. > **Note:** > -> Different tables in the same upstream database can be routed to different target databases. This kind of configuration applies only to DML and table-level DDL. -> -> - For database-level DDL such as `CREATE DATABASE`, `DROP DATABASE`, and `ALTER DATABASE`, TiCDC must be able to determine a unique target database based on the routing rules. Otherwise, synchronization of that DDL fails. -> - If you want to route different tables in the same upstream database to different target databases, you need to create the downstream target databases in advance and avoid executing upstream database-level DDL that requires automatic synchronization. +> Different tables in the same upstream database can route to different target databases. This kind of configuration applies only to DML and table-level DDL. + +> - For database-level DDL such as `CREATE DATABASE`, `DROP DATABASE`, and `ALTER DATABASE`, TiCDC must be able to determine a unique target database based on the routing rules. Otherwise, replication of that DDL fails. +> - If you want to route different tables in the same upstream database to different target databases, you need to create the downstream target databases in advance and avoid executing upstream database-level DDL that requires automatic replication. ## Configuration fields @@ -52,7 +52,7 @@ The table routing feature uses [`sink.dispatchers`](/ticdc/ticdc-changefeed-conf | Field | Description | | :--- | :--- | -| `matcher` | Matches upstream databases and tables. The syntax is the same as [table filter syntax](/table-filter.md#表库过滤语法), including wildcard matches such as `sales.*` and exclusion matches such as `!sales.tmp_*`. | +| `matcher` | Matches upstream databases and tables. The syntax is the same as [table filter syntax](/table-filter.md#syntax), including wildcard matches such as `sales.*` and exclusion matches such as `!sales.tmp_*`. | | `target-schema` | Specifies the downstream database name. If this field is not set, TiCDC keeps the upstream database name unchanged. | | `target-table` | Specifies the downstream table name. If this field is not set, TiCDC keeps the upstream table name unchanged. | @@ -130,7 +130,7 @@ Example routing results: > > This configuration applies only to database merging, not table merging. Table routing does not support merging same-named tables from multiple different databases into the same downstream table. -### Use table routing together with Kafka Sink Topic and Partition dispatchers +### Use table routing together with Kafka sink topic and partition dispatchers The same dispatcher rule can include both table routing fields and existing dispatch fields: @@ -155,13 +155,13 @@ Table routing does not change the dispatch results of Topic or Partition. The `t | Sink | Behavior | | :--- | :--- | -| MySQL Sink | DDL and DML statements are written to the routed target database and table. When the Redo feature is enabled, executing `redo apply` replays events to the routed target table. | -| Kafka Sink and Pulsar Sink | The values of the `payload` field in the protocol and the `query` field in DDL events use the routed target database and table names; the values of the `schema` and `table` fields in the encoding protocol also use the routed target database and table names. | -| Cloud Storage Sink | TiCDC generates the corresponding storage paths, schema files, table definition files, and data files based on the routed target database and table names. | +| MySQL sink | TiCDC writes DDL and DML statements to the routed target database and table. When the Redo feature is enabled, executing `redo apply` replays events to the routed target table. | +| Kafka sink and Pulsar sink | The values of the `payload` field in the protocol and the `query` field in DDL events use the routed target database and table names; the values of the `schema` and `table` fields in the encoding protocol also use the routed target database and table names. | +| Cloud storage sink | TiCDC generates the corresponding storage paths, schema files, table definition files, and data files based on the routed target database and table names. | ## DDL behavior -After table routing is enabled, TiCDC rewrites DDL statements so that the database and table names in structured DDL metadata remain consistent with those in the SQL text. +After you enable table routing, TiCDC rewrites DDL statements so that the database and table names in structured DDL metadata remain consistent with those in the SQL text. For example, if the following rule is configured: @@ -184,11 +184,12 @@ into the following downstream DDL: RENAME TABLE `archive`.`temp_table_routed` TO `archive`.`renamed_table_routed`; ``` -If a DDL statement contains table references and those table references match table routing rules, TiCDC also rewrites the referenced table names. For example, table references in `CREATE VIEW` statements and foreign key references in `ALTER TABLE` statements can be rewritten. +If a DDL statement contains table references and those table references match table routing rules, TiCDC also rewrites the referenced table names. For example, TiCDC can rewrite table references in `CREATE VIEW` statements and foreign key references in `ALTER TABLE` statements. For database-level DDL such as `CREATE DATABASE`, `DROP DATABASE`, and `ALTER DATABASE ... CHARACTER SET/COLLATE`, if the database name matches table routing rules, TiCDC rewrites the database name. **If the same upstream database matches multiple table routing rules and these rules resolve to different target database names, TiCDC cannot determine a unique target database for that database-level DDL, and the Changefeed returns a table routing error.** -When creating or updating a Changefeed, TiCDC checks for target table conflicts based on the tables currently within the synchronization scope. During runtime, TiCDC updates the conflict detection state when synchronizing DDL such as `CREATE TABLE`, `RENAME TABLE`, `DROP TABLE`, and `DROP DATABASE`. Whether a database-level DDL can determine a unique target database is evaluated when synchronizing the corresponding DDL. +When creating or updating a changefeed, TiCDC checks for target table conflicts based on the tables currently within the replication scope. During runtime, TiCDC updates the conflict detection state when replicating DDL such as `CREATE TABLE`, `RENAME TABLE`, `DROP TABLE`, and `DROP DATABASE`. TiCDC evaluates whether a database-level DDL can determine a unique target database when replicating the corresponding DDL. + ## Route conflict detection A route conflict occurs when two different upstream tables are routed to the same downstream `(schema, table)` within a single Changefeed. TiCDC does not support merging multiple upstream tables into the same downstream table. @@ -211,30 +212,30 @@ target-schema = "archive" target-table = "{table}" ``` -If both `sales.orders` and `crm.orders` are within the synchronizing scope, both tables are routed to `archive.orders`. TiCDC rejects the creation or update of the Changefeed and returns the `CDC:ErrTableRouteConflict` error. +If both `sales.orders` and `crm.orders` are within the replication scope, both tables route to `archive.orders`. TiCDC rejects the creation or update of the Changefeed and returns the `CDC:ErrTableRouteConflict` error. -While the Changefeed is running, if a DDL such as `CREATE TABLE` or `RENAME TABLE` causes two upstream tables currently within the synchronizing scope to be routed to the same target table, the Changefeed fails and returns the `CDC:ErrTableRouteConflict` error. +While the Changefeed is running, if a DDL statement such as `CREATE TABLE` or `RENAME TABLE` causes two upstream tables currently within the replication scope to route to the same target table, the Changefeed fails and returns the `CDC:ErrTableRouteConflict` error. -If an upstream table is dropped or renamed, TiCDC removes the routing relationship between that upstream table and the target table. After that, a new upstream table can continue to use that target table. However, at any given time, the same target table can correspond to only one upstream table being synchronized by the current Changefeed. +If an upstream table is dropped or renamed, TiCDC removes the routing relationship between that upstream table and the target table. After that, a new upstream table can continue to use that target table. However, at any given time, the same target table can correspond to only one upstream table being replicated by the current Changefeed. > **Warning:** > -> Route conflict detection is limited to a single Changefeed. If multiple Changefeeds write to the same downstream system, make sure that the table routing rules of these Changefeeds do not write to the same target object. +> Route conflict detection takes effect within a single changefeed. If multiple Changefeeds write to the same downstream system, make sure that the table routing rules of these Changefeeds do not write to the same target object. ## Troubleshooting | Symptom | Possible cause | Solution | | :--- | :--- | :--- | -| The `CDC:ErrInvalidTableRoutingRule` error is reported when creating or updating a Changefeed. | The `matcher` syntax is invalid, or `target-schema` or `target-table` contains unknown placeholders or mismatched braces. | Check whether `matcher` conforms to the [table filter syntax](/table-filter.md#表库过滤语法), and make sure that `target-schema` and `target-table` use only literal text, `{schema}`, and `{table}`. | -| The MQ Topic name still uses the upstream schema and table name. | Table routing does not change Topic or Partition dispatch rules. | If you need to change the Topic name, configure `topic` separately in `sink.dispatchers`. | -| The `CDC:ErrTableRoutingFailed` error is reported during DDL synchronization. | The DDL cannot be safely rewritten for table routing, or a schema-level DDL has ambiguity in the target schema. | Check the DDL type and routing rules. For schema-level DDL, make sure the same upstream schema can be parsed to only one target schema. | -| The Changefeed fails during runtime and reports the `CDC:ErrTableRouteConflict` error. | After a table is created or renamed, two different upstream tables are routed to the same downstream table. | Adjust the table routing rules or upstream DDL to ensure that, within a single Changefeed, each target table corresponds to only one upstream table being synchronized by the current Changefeed at any given time. | +| TiCDC reports the `CDC:ErrInvalidTableRoutingRule` error when you create or update a Changefeed. | The `matcher` syntax is invalid, or `target-schema` or `target-table` contains unknown placeholders or mismatched braces. | Check whether `matcher` conforms to the [table filter syntax](/table-filter.md#syntax), and make sure that `target-schema` and `target-table` use only literal text, `{schema}`, and `{table}`. | +| The MQ topic name still uses the upstream schema and table name. | Table routing does not change topic or partition dispatch rules. | If you need to change the topic name, configure `topic` separately in `sink.dispatchers`. | +| TiCDC reports the `CDC:ErrTableRoutingFailed` error during DDL replication. | TiCDC cannot safely rewrite the DDL for table routing, or a schema-level DDL has ambiguity in the target schema. | Check the DDL type and routing rules. For schema-level DDL, make sure the same upstream schema can be parsed to only one target schema. | +| The Changefeed fails during runtime and TiCDC reports the `CDC:ErrTableRouteConflict` error. | After a table is created or renamed, two different upstream tables route to the same downstream table. | Adjust the table routing rules or upstream DDL to ensure that, within a single Changefeed, each target table corresponds to only one upstream table being replicated by the current Changefeed at any given time. | ## Related documentation - [TiCDC Changefeed command-line flags and configuration parameters](/ticdc/ticdc-changefeed-config.md) - [Changefeed event filter](/ticdc/ticdc-filter.md) -- [Synchronize data to MySQL-compatible databases](/ticdc/ticdc-sink-to-mysql.md) -- [Synchronize data to Kafka](/ticdc/ticdc-sink-to-kafka.md) -- [Synchronize data to Pulsar](/ticdc/ticdc-sink-to-pulsar.md) -- [Synchronize data to storage services](/ticdc/ticdc-sink-to-cloud-storage.md) \ No newline at end of file +- [Replicate Data to MySQL-compatible Databases](/ticdc/ticdc-sink-to-mysql.md) +- [Replicate Data to Kafka](/ticdc/ticdc-sink-to-kafka.md) +- [Replicate Data to Pulsar](/ticdc/ticdc-sink-to-pulsar.md) +- [Replicate Data to Storage Services](/ticdc/ticdc-sink-to-cloud-storage.md) \ No newline at end of file From 075fa06d5701014e2e053d638fd7bc8f996678cf Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 7 Jul 2026 16:00:51 +0800 Subject: [PATCH 05/11] Apply suggestions from code review --- ticdc/ticdc-table-routing.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ticdc/ticdc-table-routing.md b/ticdc/ticdc-table-routing.md index d7623c62114af..494e5796925f3 100644 --- a/ticdc/ticdc-table-routing.md +++ b/ticdc/ticdc-table-routing.md @@ -233,8 +233,8 @@ If an upstream table is dropped or renamed, TiCDC removes the routing relationsh ## Related documentation -- [TiCDC Changefeed command-line flags and configuration parameters](/ticdc/ticdc-changefeed-config.md) -- [Changefeed event filter](/ticdc/ticdc-filter.md) +- [CLI and Configuration Parameters of TiCDC Changefeeds](/ticdc/ticdc-changefeed-config.md) +- [Changefeed Log Filters](/ticdc/ticdc-filter.md) - [Replicate Data to MySQL-compatible Databases](/ticdc/ticdc-sink-to-mysql.md) - [Replicate Data to Kafka](/ticdc/ticdc-sink-to-kafka.md) - [Replicate Data to Pulsar](/ticdc/ticdc-sink-to-pulsar.md) From 4dba47db118a4d74206725de29e5f56eb231aaff Mon Sep 17 00:00:00 2001 From: qiancai Date: Tue, 7 Jul 2026 17:00:22 +0800 Subject: [PATCH 06/11] Update ticdc-table-routing.md --- ticdc/ticdc-table-routing.md | 56 ++++++++++++++++++------------------ 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/ticdc/ticdc-table-routing.md b/ticdc/ticdc-table-routing.md index 494e5796925f3..1cb5fe54f8f66 100644 --- a/ticdc/ticdc-table-routing.md +++ b/ticdc/ticdc-table-routing.md @@ -5,17 +5,17 @@ summary: Learn about table routing configuration in the TiCDC new architecture, # TiCDC Table Routing New in v8.5.7 -TiCDC table routing allows you to map upstream tables to specified downstream database names or table names through changefeed configuration. This feature applies only to the [TiCDC new architecture](/ticdc/ticdc-architecture.md) and is not supported by the [TiCDC classic architecture](/ticdc/ticdc-classic-architecture.md). +TiCDC table routing enables you to map upstream tables to specified downstream database names or table names through changefeed configuration. This feature applies only to the [TiCDC new architecture](/ticdc/ticdc-architecture.md) and is not supported by the [TiCDC classic architecture](/ticdc/ticdc-classic-architecture.md). -Table routing changes only the database and table names that TiCDC outputs downstream. It does not change row data, column names, table schema, table filter rules, topic dispatch rules, partition dispatch rules, or column selector rules. +Table routing modifies only the database and table names that TiCDC outputs to the downstream. It does not modify row data, column names, table schema, table filter rules, topic dispatch rules, partition dispatch rules, or column selector rules. ## Usage scenarios -Table routing is suitable for the following scenarios: +You can configure table routing in the following scenarios: - Replicating `sales.orders` to `archive.sales_orders`, or to a table that follows other downstream naming conventions. - Replicating multiple source databases to the same target database while keeping target table names unique to distinguish the source, for example, replicating `tenant_001.orders` to `tenant_mirror.tenant_001_orders`. -- Building migration, disaster recovery, archiving, or shadow Changefeeds to avoid writing to downstream objects with the same names as upstream ones. +- Building migration, disaster recovery, archiving, or shadow changefeeds to avoid writing to downstream objects with the same names as upstream ones. - Exposing stable database and table names to MQ consumer applications or storage services. > **Note:** @@ -25,7 +25,7 @@ Table routing is suitable for the following scenarios: ## Configure table routing -Before configuring table routing, enable the TiCDC new architecture. For details, see [`newarch`](/ticdc-server-config.md#newarch-new-in-v854-release1). +Before configuring table routing, enable the TiCDC new architecture. For more information, see [`newarch`](/ticdc-server-config.md#newarch-new-in-v854-release1). The following example routes `sales.orders` to `archive.sales_orders`: @@ -41,14 +41,14 @@ After the changefeed starts, TiCDC writes the DML and DDL events of `sales.order > **Note:** > -> Different tables in the same upstream database can route to different target databases. This kind of configuration applies only to DML and table-level DDL. +> Different tables in the same upstream database can route to different target databases, but this routing configuration applies only to DML and table-level DDL. > - For database-level DDL such as `CREATE DATABASE`, `DROP DATABASE`, and `ALTER DATABASE`, TiCDC must be able to determine a unique target database based on the routing rules. Otherwise, replication of that DDL fails. > - If you want to route different tables in the same upstream database to different target databases, you need to create the downstream target databases in advance and avoid executing upstream database-level DDL that requires automatic replication. ## Configuration fields -The table routing feature uses [`sink.dispatchers`](/ticdc/ticdc-changefeed-config.md#dispatchers) as the configuration entry. +To use the table routing feature, you can configure the following fields in [`sink.dispatchers`](/ticdc/ticdc-changefeed-config.md#dispatchers): | Field | Description | | :--- | :--- | @@ -58,10 +58,10 @@ The table routing feature uses [`sink.dispatchers`](/ticdc/ticdc-changefeed-conf The matching behavior is as follows: -- Only `sink.dispatchers` rules with `target-schema` or `target-table` set participate in table routing. +- Only `sink.dispatchers` rules that specify `target-schema` or `target-table` are used for table routing. - If a table matches multiple table routing rules, only the first matching rule in `sink.dispatchers` takes effect. - `matcher` always matches upstream database and table names, not the routed target database and table names. -- The Changefeed configuration item `case-sensitive` affects only whether the `matcher` in table routing is case-sensitive. It does not rewrite the expansion results of `{schema}` and `{table}`. For details, see [`case-sensitive`](/ticdc/ticdc-changefeed-config.md#case-sensitive). +- The changefeed configuration item `case-sensitive` affects only whether the `matcher` in table routing is case-sensitive. It does not rewrite the expansion results of `{schema}` and `{table}`. For more information, see [`case-sensitive`](/ticdc/ticdc-changefeed-config.md#case-sensitive). ### Placeholders @@ -72,7 +72,7 @@ You can use the following placeholders in `target-schema` and `target-table`: | `{schema}` | The upstream database name, preserving the actual matched database name case. | | `{table}` | The upstream table name, preserving the actual matched table name case. | -The values of `target-schema` and `target-table` can contain only literal text, `{schema}`, and `{table}`. If you use an unknown placeholder such as `{db}`, TiCDC rejects the Changefeed configuration and returns the `CDC:ErrInvalidTableRoutingRule` error. +The values of `target-schema` and `target-table` can contain only literal text, `{schema}`, and `{table}`. If you use an unknown placeholder such as `{db}`, TiCDC rejects the changefeed configuration and returns the `CDC:ErrInvalidTableRoutingRule` error. Using the source table `sales.orders` as an example: @@ -107,7 +107,7 @@ Example routing results: ### Route multiple databases to the same target database -When routing multiple databases to the same target database, it is recommended to include `{schema}` in `target-table` to ensure unique target table names. +To route multiple databases to the same target database, it is recommended to include `{schema}` in `target-table` to ensure unique target table names. ```toml [filter] @@ -128,11 +128,11 @@ Example routing results: > **Note:** > -> This configuration applies only to database merging, not table merging. Table routing does not support merging same-named tables from multiple different databases into the same downstream table. +> This configuration applies only to database merging, not table merging. Table routing does not support merging tables with the same name from multiple upstream databases into the same downstream table. ### Use table routing together with Kafka sink topic and partition dispatchers -The same dispatcher rule can include both table routing fields and existing dispatch fields: +You can configure both table routing fields and existing dispatch fields in the same dispatcher rule: ```toml [filter] @@ -147,16 +147,16 @@ target-schema = "public" target-table = "orders" ``` -In the preceding example, table routing changes the database and table names exposed in downstream data to `public.orders`. +In the preceding example, table routing modifies the database and table names exposed in downstream data to `public.orders`. -Table routing does not change the dispatch results of Topic or Partition. The `topic` and `partition` dispatchers still perform matching and dispatch calculation based on the upstream table `sales.orders`. +Table routing does not change the dispatch results of topic or partition. The `topic` and `partition` dispatchers still perform matching and dispatch calculation based on the upstream table `sales.orders`. ## Output behavior | Sink | Behavior | | :--- | :--- | | MySQL sink | TiCDC writes DDL and DML statements to the routed target database and table. When the Redo feature is enabled, executing `redo apply` replays events to the routed target table. | -| Kafka sink and Pulsar sink | The values of the `payload` field in the protocol and the `query` field in DDL events use the routed target database and table names; the values of the `schema` and `table` fields in the encoding protocol also use the routed target database and table names. | +| Kafka sink and Pulsar sink | The values of the `payload` field in the protocol and the `query` field in DDL events use the routed target database and table names. The values of the `schema` and `table` fields in the encoding protocol also use the routed target database and table names. | | Cloud storage sink | TiCDC generates the corresponding storage paths, schema files, table definition files, and data files based on the routed target database and table names. | ## DDL behavior @@ -178,7 +178,7 @@ TiCDC rewrites the following upstream DDL: RENAME TABLE `sales`.`temp_table` TO `sales`.`renamed_table`; ``` -into the following downstream DDL: +Into the following downstream DDL: ```sql RENAME TABLE `archive`.`temp_table_routed` TO `archive`.`renamed_table_routed`; @@ -186,13 +186,13 @@ RENAME TABLE `archive`.`temp_table_routed` TO `archive`.`renamed_table_routed`; If a DDL statement contains table references and those table references match table routing rules, TiCDC also rewrites the referenced table names. For example, TiCDC can rewrite table references in `CREATE VIEW` statements and foreign key references in `ALTER TABLE` statements. -For database-level DDL such as `CREATE DATABASE`, `DROP DATABASE`, and `ALTER DATABASE ... CHARACTER SET/COLLATE`, if the database name matches table routing rules, TiCDC rewrites the database name. **If the same upstream database matches multiple table routing rules and these rules resolve to different target database names, TiCDC cannot determine a unique target database for that database-level DDL, and the Changefeed returns a table routing error.** +For database-level DDL such as `CREATE DATABASE`, `DROP DATABASE`, and `ALTER DATABASE ... CHARACTER SET/COLLATE`, if the database name matches table routing rules, TiCDC rewrites the database name. **If the same upstream database matches multiple table routing rules and these rules map to different target database names, TiCDC cannot determine a unique target database for that database-level DDL, and the changefeed returns a table routing error.** -When creating or updating a changefeed, TiCDC checks for target table conflicts based on the tables currently within the replication scope. During runtime, TiCDC updates the conflict detection state when replicating DDL such as `CREATE TABLE`, `RENAME TABLE`, `DROP TABLE`, and `DROP DATABASE`. TiCDC evaluates whether a database-level DDL can determine a unique target database when replicating the corresponding DDL. +When creating or updating a changefeed, TiCDC checks for target table conflicts based on the tables currently within the replication scope. During runtime, TiCDC updates the conflict detection state when replicating DDL such as `CREATE TABLE`, `RENAME TABLE`, `DROP TABLE`, and `DROP DATABASE`. For a database-level DDL statement, TiCDC evaluates whether it can determine a unique target database when replicating it. ## Route conflict detection -A route conflict occurs when two different upstream tables are routed to the same downstream `(schema, table)` within a single Changefeed. TiCDC does not support merging multiple upstream tables into the same downstream table. +A route conflict occurs when two different upstream tables are routed to the same downstream `(schema, table)` within a single changefeed. TiCDC does not support merging multiple upstream tables into the same downstream table. For example, the following configuration might cause a conflict: @@ -212,24 +212,24 @@ target-schema = "archive" target-table = "{table}" ``` -If both `sales.orders` and `crm.orders` are within the replication scope, both tables route to `archive.orders`. TiCDC rejects the creation or update of the Changefeed and returns the `CDC:ErrTableRouteConflict` error. +If both `sales.orders` and `crm.orders` are within the replication scope, both tables route to `archive.orders`. TiCDC rejects the creation or update of the changefeed and returns the `CDC:ErrTableRouteConflict` error. -While the Changefeed is running, if a DDL statement such as `CREATE TABLE` or `RENAME TABLE` causes two upstream tables currently within the replication scope to route to the same target table, the Changefeed fails and returns the `CDC:ErrTableRouteConflict` error. +While the changefeed is running, if a DDL statement such as `CREATE TABLE` or `RENAME TABLE` causes two upstream tables currently within the replication scope to route to the same target table, the changefeed fails and returns the `CDC:ErrTableRouteConflict` error. -If an upstream table is dropped or renamed, TiCDC removes the routing relationship between that upstream table and the target table. After that, a new upstream table can continue to use that target table. However, at any given time, the same target table can correspond to only one upstream table being replicated by the current Changefeed. +If an upstream table is dropped or renamed, TiCDC removes the routing relationship between that upstream table and the target table. After that, a new upstream table can continue to use that target table. However, at any given time, the same target table can correspond to only one upstream table being replicated by the current changefeed. > **Warning:** > -> Route conflict detection takes effect within a single changefeed. If multiple Changefeeds write to the same downstream system, make sure that the table routing rules of these Changefeeds do not write to the same target object. +> Route conflict detection takes effect within a single changefeed. If multiple changefeeds write to the same downstream system, make sure that the table routing rules of these changefeeds do not write to the same target object. ## Troubleshooting | Symptom | Possible cause | Solution | | :--- | :--- | :--- | -| TiCDC reports the `CDC:ErrInvalidTableRoutingRule` error when you create or update a Changefeed. | The `matcher` syntax is invalid, or `target-schema` or `target-table` contains unknown placeholders or mismatched braces. | Check whether `matcher` conforms to the [table filter syntax](/table-filter.md#syntax), and make sure that `target-schema` and `target-table` use only literal text, `{schema}`, and `{table}`. | -| The MQ topic name still uses the upstream schema and table name. | Table routing does not change topic or partition dispatch rules. | If you need to change the topic name, configure `topic` separately in `sink.dispatchers`. | -| TiCDC reports the `CDC:ErrTableRoutingFailed` error during DDL replication. | TiCDC cannot safely rewrite the DDL for table routing, or a schema-level DDL has ambiguity in the target schema. | Check the DDL type and routing rules. For schema-level DDL, make sure the same upstream schema can be parsed to only one target schema. | -| The Changefeed fails during runtime and TiCDC reports the `CDC:ErrTableRouteConflict` error. | After a table is created or renamed, two different upstream tables route to the same downstream table. | Adjust the table routing rules or upstream DDL to ensure that, within a single Changefeed, each target table corresponds to only one upstream table being replicated by the current Changefeed at any given time. | +| TiCDC reports the `CDC:ErrInvalidTableRoutingRule` error when you create or update a changefeed. | The `matcher` syntax is invalid, or `target-schema` or `target-table` contains unknown placeholders or mismatched braces. | Check whether `matcher` conforms to the [table filter syntax](/table-filter.md#syntax), and make sure that `target-schema` and `target-table` use only literal text, `{schema}`, and `{table}`. | +| The MQ topic name still uses the upstream database and table name. | Table routing does not change topic or partition dispatch rules. | If you need to change the topic name, configure `topic` separately in `sink.dispatchers`. | +| TiCDC reports the `CDC:ErrTableRoutingFailed` error during DDL replication. | TiCDC cannot safely rewrite the DDL for table routing, or the target database of a database-level DDL is ambiguous. | Check the DDL type and routing rules. For database-level DDL, make sure that the same upstream database maps to only one target database. | +| The changefeed fails during runtime and TiCDC reports the `CDC:ErrTableRouteConflict` error. | After a table is created or renamed, two different upstream tables route to the same downstream table. | Adjust the table routing rules or upstream DDL to ensure that, within a single changefeed, each target table corresponds to only one upstream table being replicated by the current changefeed at any given time. | ## Related documentation From 06a74fc2182460ae1517237d0253fecd84c418ae Mon Sep 17 00:00:00 2001 From: qiancai Date: Tue, 7 Jul 2026 17:27:29 +0800 Subject: [PATCH 07/11] minor wording updates --- ticdc/ticdc-changefeed-config.md | 2 +- ticdc/ticdc-table-routing.md | 36 ++++++++++++++++---------------- 2 files changed, 19 insertions(+), 19 deletions(-) diff --git a/ticdc/ticdc-changefeed-config.md b/ticdc/ticdc-changefeed-config.md index 9d2f510ccfcd1..d545abcfca72c 100644 --- a/ticdc/ticdc-changefeed-config.md +++ b/ticdc/ticdc-changefeed-config.md @@ -182,7 +182,7 @@ For more information, see [Event filter rules](/ticdc/ticdc-filter.md#event-filt #### `dispatchers` -- In versions earlier than v8.5.7, you can use `dispatchers` to configure event dispatchers, and it takes effect only when the downstream is an MQ sink. Starting from v8.5.7, for the [new TiCDC architecture](/ticdc/ticdc-architecture.md), you can also use `dispatchers` to configure table routing, mapping upstream tables to specified downstream database names or table names. For more information, see [TiCDC table routing](/ticdc/ticdc-table-routing.md). +- In versions earlier than v8.5.7, you can use `dispatchers` to configure event dispatchers, and `dispatchers` takes effect only when the downstream is an MQ sink. Starting from v8.5.7, when you use the [new TiCDC architecture](/ticdc/ticdc-architecture.md), you can also use `dispatchers` to configure table routing, mapping upstream tables to specific downstream database or table names. For more information, see [TiCDC table routing](/ticdc/ticdc-table-routing.md). - Starting from v6.1.0, TiDB supports two types of event dispatchers: partition and topic. - The matching syntax of matcher is the same as the filter rule syntax. - When the downstream MQ is Pulsar, if the routing rule for `partition` is not specified as any of `ts`, `index-value`, `table`, or `default`, each Pulsar message will be routed using the string you set as the key. For example, if you specify the routing rule for a matcher as the string `code`, then all Pulsar messages that match that matcher will be routed with `code` as the key. diff --git a/ticdc/ticdc-table-routing.md b/ticdc/ticdc-table-routing.md index 1cb5fe54f8f66..bddce0877ba75 100644 --- a/ticdc/ticdc-table-routing.md +++ b/ticdc/ticdc-table-routing.md @@ -5,7 +5,7 @@ summary: Learn about table routing configuration in the TiCDC new architecture, # TiCDC Table Routing New in v8.5.7 -TiCDC table routing enables you to map upstream tables to specified downstream database names or table names through changefeed configuration. This feature applies only to the [TiCDC new architecture](/ticdc/ticdc-architecture.md) and is not supported by the [TiCDC classic architecture](/ticdc/ticdc-classic-architecture.md). +TiCDC table routing enables you to map upstream tables to specific downstream database names or table names through changefeed configuration. This feature applies only to the [TiCDC new architecture](/ticdc/ticdc-architecture.md) and is not supported by the [TiCDC classic architecture](/ticdc/ticdc-classic-architecture.md). Table routing modifies only the database and table names that TiCDC outputs to the downstream. It does not modify row data, column names, table schema, table filter rules, topic dispatch rules, partition dispatch rules, or column selector rules. @@ -14,14 +14,14 @@ Table routing modifies only the database and table names that TiCDC outputs to t You can configure table routing in the following scenarios: - Replicating `sales.orders` to `archive.sales_orders`, or to a table that follows other downstream naming conventions. -- Replicating multiple source databases to the same target database while keeping target table names unique to distinguish the source, for example, replicating `tenant_001.orders` to `tenant_mirror.tenant_001_orders`. -- Building migration, disaster recovery, archiving, or shadow changefeeds to avoid writing to downstream objects with the same names as upstream ones. +- Replicating multiple source databases to the same target database while keeping target table names unique to distinguish the source database, for example, replicating `tenant_001.orders` to `tenant_mirror.tenant_001_orders`. +- Building changefeeds for migration, disaster recovery, archiving, or shadow traffic to avoid writing to downstream objects with the same names as upstream ones. - Exposing stable database and table names to MQ consumer applications or storage services. > **Note:** > -> Table routing supports only one-to-one mapping from an upstream table to a downstream target table. It does not support merging multiple upstream tables into the same downstream table. -> Table routing does not support splitting one upstream table into multiple downstream tables, nor transforming row data content. +> Table routing supports only a one-to-one mapping from an upstream table to a downstream target table. It does not support merging multiple upstream tables into the same downstream table. +> Table routing does not support splitting one upstream table into multiple downstream tables, or transforming row data content. ## Configure table routing @@ -37,11 +37,11 @@ target-schema = "archive" target-table = "{schema}_{table}" ``` -After the changefeed starts, TiCDC writes the DML and DDL events of `sales.orders` to the downstream `archive.sales_orders`. +After the changefeed starts, TiCDC writes the DML and DDL events from `sales.orders` to the downstream `archive.sales_orders`. > **Note:** > -> Different tables in the same upstream database can route to different target databases, but this routing configuration applies only to DML and table-level DDL. +> Different tables in the same upstream database can be routed to different target databases, but this routing configuration applies only to DML and table-level DDL. > - For database-level DDL such as `CREATE DATABASE`, `DROP DATABASE`, and `ALTER DATABASE`, TiCDC must be able to determine a unique target database based on the routing rules. Otherwise, replication of that DDL fails. > - If you want to route different tables in the same upstream database to different target databases, you need to create the downstream target databases in advance and avoid executing upstream database-level DDL that requires automatic replication. @@ -52,7 +52,7 @@ To use the table routing feature, you can configure the following fields in [`si | Field | Description | | :--- | :--- | -| `matcher` | Matches upstream databases and tables. The syntax is the same as [table filter syntax](/table-filter.md#syntax), including wildcard matches such as `sales.*` and exclusion matches such as `!sales.tmp_*`. | +| `matcher` | Matches upstream databases and tables. The syntax is the same as the [table filter syntax](/table-filter.md#syntax), including wildcard matches such as `sales.*` and exclusion matches such as `!sales.tmp_*`. | | `target-schema` | Specifies the downstream database name. If this field is not set, TiCDC keeps the upstream database name unchanged. | | `target-table` | Specifies the downstream table name. If this field is not set, TiCDC keeps the upstream table name unchanged. | @@ -61,7 +61,7 @@ The matching behavior is as follows: - Only `sink.dispatchers` rules that specify `target-schema` or `target-table` are used for table routing. - If a table matches multiple table routing rules, only the first matching rule in `sink.dispatchers` takes effect. - `matcher` always matches upstream database and table names, not the routed target database and table names. -- The changefeed configuration item `case-sensitive` affects only whether the `matcher` in table routing is case-sensitive. It does not rewrite the expansion results of `{schema}` and `{table}`. For more information, see [`case-sensitive`](/ticdc/ticdc-changefeed-config.md#case-sensitive). +- The changefeed configuration item `case-sensitive` affects only whether the `matcher` in table routing is case-sensitive. It does not change the case of values expanded from `{schema}` and `{table}`. For more information, see [`case-sensitive`](/ticdc/ticdc-changefeed-config.md#case-sensitive). ### Placeholders @@ -69,8 +69,8 @@ You can use the following placeholders in `target-schema` and `target-table`: | Placeholder | Description | | :--- | :--- | -| `{schema}` | The upstream database name, preserving the actual matched database name case. | -| `{table}` | The upstream table name, preserving the actual matched table name case. | +| `{schema}` | The upstream database name, preserving the case of the actual matched database name. | +| `{table}` | The upstream table name, preserving the case of the actual matched table name. | The values of `target-schema` and `target-table` can contain only literal text, `{schema}`, and `{table}`. If you use an unknown placeholder such as `{db}`, TiCDC rejects the changefeed configuration and returns the `CDC:ErrInvalidTableRoutingRule` error. @@ -184,11 +184,11 @@ Into the following downstream DDL: RENAME TABLE `archive`.`temp_table_routed` TO `archive`.`renamed_table_routed`; ``` -If a DDL statement contains table references and those table references match table routing rules, TiCDC also rewrites the referenced table names. For example, TiCDC can rewrite table references in `CREATE VIEW` statements and foreign key references in `ALTER TABLE` statements. +If a DDL statement contains table references and the referenced tables match table routing rules, TiCDC also rewrites the referenced table names. For example, TiCDC can rewrite table references in `CREATE VIEW` statements and foreign key references in `ALTER TABLE` statements. For database-level DDL such as `CREATE DATABASE`, `DROP DATABASE`, and `ALTER DATABASE ... CHARACTER SET/COLLATE`, if the database name matches table routing rules, TiCDC rewrites the database name. **If the same upstream database matches multiple table routing rules and these rules map to different target database names, TiCDC cannot determine a unique target database for that database-level DDL, and the changefeed returns a table routing error.** -When creating or updating a changefeed, TiCDC checks for target table conflicts based on the tables currently within the replication scope. During runtime, TiCDC updates the conflict detection state when replicating DDL such as `CREATE TABLE`, `RENAME TABLE`, `DROP TABLE`, and `DROP DATABASE`. For a database-level DDL statement, TiCDC evaluates whether it can determine a unique target database when replicating it. +When creating or updating a changefeed, TiCDC checks for target table conflicts based on the tables currently in the replication scope. At runtime, TiCDC updates the conflict detection state when replicating DDL such as `CREATE TABLE`, `RENAME TABLE`, `DROP TABLE`, and `DROP DATABASE`. For a database-level DDL statement, TiCDC evaluates whether it can determine a unique target database when replicating it. ## Route conflict detection @@ -212,15 +212,15 @@ target-schema = "archive" target-table = "{table}" ``` -If both `sales.orders` and `crm.orders` are within the replication scope, both tables route to `archive.orders`. TiCDC rejects the creation or update of the changefeed and returns the `CDC:ErrTableRouteConflict` error. +If both `sales.orders` and `crm.orders` are within the replication scope, both tables are routed to `archive.orders`. TiCDC rejects the creation or update of the changefeed and returns the `CDC:ErrTableRouteConflict` error. -While the changefeed is running, if a DDL statement such as `CREATE TABLE` or `RENAME TABLE` causes two upstream tables currently within the replication scope to route to the same target table, the changefeed fails and returns the `CDC:ErrTableRouteConflict` error. +While the changefeed is running, if a DDL statement such as `CREATE TABLE` or `RENAME TABLE` causes two upstream tables currently within the replication scope to be routed to the same target table, the changefeed fails and returns the `CDC:ErrTableRouteConflict` error. If an upstream table is dropped or renamed, TiCDC removes the routing relationship between that upstream table and the target table. After that, a new upstream table can continue to use that target table. However, at any given time, the same target table can correspond to only one upstream table being replicated by the current changefeed. > **Warning:** > -> Route conflict detection takes effect within a single changefeed. If multiple changefeeds write to the same downstream system, make sure that the table routing rules of these changefeeds do not write to the same target object. +> Route conflict detection takes effect within a single changefeed. If multiple changefeeds write to the same downstream system, make sure that the table routing rules of these changefeeds do not route tables to the same target object. ## Troubleshooting @@ -228,8 +228,8 @@ If an upstream table is dropped or renamed, TiCDC removes the routing relationsh | :--- | :--- | :--- | | TiCDC reports the `CDC:ErrInvalidTableRoutingRule` error when you create or update a changefeed. | The `matcher` syntax is invalid, or `target-schema` or `target-table` contains unknown placeholders or mismatched braces. | Check whether `matcher` conforms to the [table filter syntax](/table-filter.md#syntax), and make sure that `target-schema` and `target-table` use only literal text, `{schema}`, and `{table}`. | | The MQ topic name still uses the upstream database and table name. | Table routing does not change topic or partition dispatch rules. | If you need to change the topic name, configure `topic` separately in `sink.dispatchers`. | -| TiCDC reports the `CDC:ErrTableRoutingFailed` error during DDL replication. | TiCDC cannot safely rewrite the DDL for table routing, or the target database of a database-level DDL is ambiguous. | Check the DDL type and routing rules. For database-level DDL, make sure that the same upstream database maps to only one target database. | -| The changefeed fails during runtime and TiCDC reports the `CDC:ErrTableRouteConflict` error. | After a table is created or renamed, two different upstream tables route to the same downstream table. | Adjust the table routing rules or upstream DDL to ensure that, within a single changefeed, each target table corresponds to only one upstream table being replicated by the current changefeed at any given time. | +| TiCDC reports the `CDC:ErrTableRoutingFailed` error during DDL replication. | TiCDC cannot safely rewrite the DDL for table routing, or the target database for a database-level DDL is ambiguous. | Check the DDL type and routing rules. For database-level DDL, make sure that the same upstream database maps to only one target database. | +| The changefeed fails during runtime and TiCDC reports the `CDC:ErrTableRouteConflict` error. | After a table is created or renamed, two different upstream tables are routed to the same downstream table. | Adjust the table routing rules or upstream DDL to ensure that, within a single changefeed, each target table corresponds to only one upstream table being replicated by the current changefeed at any given time. | ## Related documentation From c2af3ed91a5b906b54d932a8ae8a74efad25cd59 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 7 Jul 2026 20:04:57 +0800 Subject: [PATCH 08/11] fix a broken link --- ticdc/ticdc-table-routing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ticdc/ticdc-table-routing.md b/ticdc/ticdc-table-routing.md index bddce0877ba75..5c10e9e01b314 100644 --- a/ticdc/ticdc-table-routing.md +++ b/ticdc/ticdc-table-routing.md @@ -25,7 +25,7 @@ You can configure table routing in the following scenarios: ## Configure table routing -Before configuring table routing, enable the TiCDC new architecture. For more information, see [`newarch`](/ticdc-server-config.md#newarch-new-in-v854-release1). +Before configuring table routing, enable the TiCDC new architecture. For more information, see [`newarch`](/ticdc/ticdc-server-config.md#newarch-new-in-v854-release1). The following example routes `sales.orders` to `archive.sales_orders`: From bc89772f937ec60b8885b6d57afa284942dfc776 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 7 Jul 2026 22:03:17 +0800 Subject: [PATCH 09/11] Apply suggestions from code review --- ticdc/ticdc-table-routing.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ticdc/ticdc-table-routing.md b/ticdc/ticdc-table-routing.md index 5c10e9e01b314..310eaab7053c3 100644 --- a/ticdc/ticdc-table-routing.md +++ b/ticdc/ticdc-table-routing.md @@ -172,13 +172,13 @@ target-schema = "archive" target-table = "{table}_routed" ``` -TiCDC rewrites the following upstream DDL: +For the following upstream DDL: ```sql RENAME TABLE `sales`.`temp_table` TO `sales`.`renamed_table`; ``` -Into the following downstream DDL: +TiCDC rewrites it into the following downstream DDL: ```sql RENAME TABLE `archive`.`temp_table_routed` TO `archive`.`renamed_table_routed`; From 87575b8b09467141eaf480d4c69daab24c5169be Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 7 Jul 2026 22:04:34 +0800 Subject: [PATCH 10/11] Update ticdc/ticdc-table-routing.md --- ticdc/ticdc-table-routing.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ticdc/ticdc-table-routing.md b/ticdc/ticdc-table-routing.md index 310eaab7053c3..b0b4f3070732b 100644 --- a/ticdc/ticdc-table-routing.md +++ b/ticdc/ticdc-table-routing.md @@ -172,13 +172,13 @@ target-schema = "archive" target-table = "{table}_routed" ``` -For the following upstream DDL: +For the following upstream DDL statement: ```sql RENAME TABLE `sales`.`temp_table` TO `sales`.`renamed_table`; ``` -TiCDC rewrites it into the following downstream DDL: +TiCDC rewrites it into the following downstream DDL statement: ```sql RENAME TABLE `archive`.`temp_table_routed` TO `archive`.`renamed_table_routed`; From de60bcc44127d2387ff8cdc64fbeb152da155ff6 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 7 Jul 2026 22:22:02 +0800 Subject: [PATCH 11/11] Update ticdc/ticdc-changefeed-config.md --- ticdc/ticdc-changefeed-config.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ticdc/ticdc-changefeed-config.md b/ticdc/ticdc-changefeed-config.md index d545abcfca72c..4c1c419f33555 100644 --- a/ticdc/ticdc-changefeed-config.md +++ b/ticdc/ticdc-changefeed-config.md @@ -182,7 +182,7 @@ For more information, see [Event filter rules](/ticdc/ticdc-filter.md#event-filt #### `dispatchers` -- In versions earlier than v8.5.7, you can use `dispatchers` to configure event dispatchers, and `dispatchers` takes effect only when the downstream is an MQ sink. Starting from v8.5.7, when you use the [new TiCDC architecture](/ticdc/ticdc-architecture.md), you can also use `dispatchers` to configure table routing, mapping upstream tables to specific downstream database or table names. For more information, see [TiCDC table routing](/ticdc/ticdc-table-routing.md). +- When the changefeed downstream is an MQ sink, you can use `dispatchers` to configure event dispatchers. Starting from v8.5.7, for the [new TiCDC architecture](/ticdc/ticdc-architecture.md), you can also use `dispatchers` to configure table routing, mapping upstream tables to specific downstream database or table names. For more information, see [TiCDC table routing](/ticdc/ticdc-table-routing.md). - Starting from v6.1.0, TiDB supports two types of event dispatchers: partition and topic. - The matching syntax of matcher is the same as the filter rule syntax. - When the downstream MQ is Pulsar, if the routing rule for `partition` is not specified as any of `ts`, `index-value`, `table`, or `default`, each Pulsar message will be routed using the string you set as the key. For example, if you specify the routing rule for a matcher as the string `code`, then all Pulsar messages that match that matcher will be routed with `code` as the key.