Skip to content

[Fix] Swap canConvert argument order in CdcActionCommonUtils.schemaCompatible#8369

Open
0dunay0 wants to merge 1 commit into
apache:masterfrom
0dunay0:fix/cdc-schema-compat-canconvert-order
Open

[Fix] Swap canConvert argument order in CdcActionCommonUtils.schemaCompatible#8369
0dunay0 wants to merge 1 commit into
apache:masterfrom
0dunay0:fix/cdc-schema-compat-canconvert-order

Conversation

@0dunay0

@0dunay0 0dunay0 commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes a bug in CdcActionCommonUtils.schemaCompatible() where valid type pairings were incorrectly rejected at CDC job startup.

Root Cause

The original code called canConvert(sourceType, paimonType) asking "can the source type be directly written into Paimon?" For a Paimon INT column against a BIGINT source, this returns IGNORE (narrowing not permitted), blocking the job even though Paimon can safely evolve INT -> BIGINT.

Fix

canConvert(old, new) returns one of:

  • CONVERT — old can be widened to new (evolution needed but valid)
  • IGNORE — old is already broader than new (source fits as-is, no evolution needed)
  • EXCEPTION — incompatible type families

A pairing is compatible if either direction produces CONVERT:

ConvertAction sourceToPaimon = canConvert(sourceType, paimonType, mapping);
ConvertAction paimonToSource = canConvert(paimonType, sourceType, mapping);

if (sourceToPaimon != CONVERT && paimonToSource != CONVERT) {
    return false;
}

This covers both valid cases:

  1. Source fits in existing Paimon type as-is (sourceToPaimon == CONVERT)
  2. Paimon can evolve to accommodate the source (paimonToSource == CONVERT)

Cross-family pairs (e.g. INT vs STRING) return EXCEPTION in both directions so neither check produces CONVERT and they are correctly rejected.

Cases

Paimon Source sourceToPaimon paimonToSource Result
INT BIGINT IGNORE CONVERT compatible — Paimon can evolve
STRING CHAR(10) CONVERT IGNORE compatible — source fits as-is
BIGINT INT CONVERT IGNORE compatible — source fits as-is
INT STRING EXCEPTION EXCEPTION incompatible

Verifying this change

Added testSchemaCompatibleTypeWidening to SchemaEvolutionTest covering all four cases above.

Closes #5640

@0dunay0 0dunay0 marked this pull request as draft June 27, 2026 23:52
…mpatible

schemaCompatible() was calling canConvert(sourceType, paimonType) but
the method signature is canConvert(oldType, newType), where old is the
existing Paimon type and new is the incoming source type.

This caused valid type widenings such as INT -> BIGINT to be rejected
at job startup with 'Paimon schema and source table schema are not
compatible', even though the same widening is handled correctly at
runtime by UpdatedDataFieldsProcessFunction.

Fix: swap the arguments and update the log message to reflect the
correct direction of the conversion check.

Closes apache#5640
@0dunay0 0dunay0 force-pushed the fix/cdc-schema-compat-canconvert-order branch from bf8991a to a1d95fa Compare June 27, 2026 23:59
@0dunay0 0dunay0 marked this pull request as ready for review June 28, 2026 00:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Flink CDC Schema Compatibility Check: Incorrect Argument Order in canConvert Call

1 participant