[fix](cdc) move SSL/JDBC config before startup mode to fix offset on SSL-required MySQL

[maks3201]

What problem does this PR solve?

Issue Number: close #66558

Related PR: N/A

Problem Summary:

In MySqlSourceReader.generateMySqlConfig(), the startup mode block (handling offset=latest, earliest, and timestamp) calls initializeEffectiveOffset(), which opens a JDBC connection to the source MySQL to resolve the current binlog filename and position. However, SSL properties (ssl_mode, ssl_rootcert) and JDBC connection properties are applied after this block.

This means on any MySQL instance configured with require_secure_transport=ON, the JDBC connection inside initializeEffectiveOffset() attempts a plaintext connection, which the server rejects. The streaming job fails immediately at startup.

The fix moves the JDBC properties and SSL configuration block to before the startup mode switch block. This ensures initializeEffectiveOffset() uses the correct SSL settings when building its temporary connection.

Why the reordering is safe for non-SSL users: When ssl_mode is not specified in the job properties, the SSL if-blocks are skipped entirely and jdbcProperties contains only the URL connection parameters (same as before). The JDBC connection inside initializeEffectiveOffset() behaves identically whether the properties object is set on the factory before or after the startup mode block — the only difference is that now it is available at the point it is needed.

Affected offset modes:

• latest — calls initializeEffectiveOffset() ✗ fails without this fix
• earliest — calls initializeEffectiveOffset() ✗ fails without this fix
• timestamp (13-digit epoch) — calls initializeEffectiveOffset() ✗ fails without this fix
• initial / snapshot — do NOT call initializeEffectiveOffset() → unaffected

Release note

Fixed a bug where CDC streaming jobs with offset=latest, offset=earliest, or a timestamp offset failed immediately on MySQL instances that require SSL (require_secure_transport=ON). The SSL connection properties were being configured after the code that needed them.

Check List (For Author)

• Test

  • No regression test included

  Note on testing: The defect is a method-call ordering issue within a single method. A unit test cannot meaningfully assert that JDBC properties are set on the factory before initializeEffectiveOffset() is called without fragile mocking of internal sequencing. A meaningful regression test requires a live MySQL instance with require_secure_transport=ON, which is not available in upstream CI. The fix has been validated manually against Aurora MySQL 8.0 with SSL required.

• If it changes/adds/deletes any user-facing features or behaviours (NO — the fix restores intended behaviour, no new features):

  • N/A — no docs or audit log changes needed

• If it affects the operation/behaviour of existing data (NO):

  • N/A — no data migration needed

---

Backport: Please label dev/4.1.x. This bug is present in 4.1.2 and 4.1.3. The fix applies cleanly to branch-4.1.

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[maks3201]

This bug is present in released versions 4.1.2 and 4.1.3. Could a committer please apply the dev/4.1.x label to trigger the auto-cherry-pick to branch-4.1? I'm a first-time contributor and cannot apply labels myself. Thank you!

[maks3201]

Thanks for the triage. You were right that the fix direction was correct but the PR had no regression test, so I have added one: MySqlStartupSslITCase.

Following your suggestion it starts a MySQL container, turns on require_secure_transport=ON, and runs a CDC job with offset=latest and ssl_mode=require, which is the mode that reaches initializeEffectiveOffset().

I verified it in both directions, because a regression test that passes on patched and unpatched code alike would be worthless.

On this PR's code the test passes:

INFO: SSL enabled
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 56.82 s -- in org.apache.doris.cdcclient.itcase.MySqlStartupSslITCase
[INFO] BUILD SUCCESS

With only MySqlSourceReader.java reverted to master, the test fails, and for the right reason:

[ERROR] Tests run: 3, Failures: 0, Errors: 1, Skipped: 0 <<< FAILURE! -- in MySqlStartupSslITCase
org.apache.flink.util.FlinkRuntimeException: Cannot read the binlog filename and position via 'SHOW MASTER STATUS'
    at org.apache.flink.cdc.connectors.mysql.debezium.DebeziumUtils.currentBinlogOffset(DebeziumUtils.java:143)
    at org.apache.flink.cdc.connectors.mysql.source.offset.BinlogOffsetUtils.initializeEffectiveOffset(BinlogOffsetUtils.java:58)
    at org.apache.doris.cdcclient.source.reader.mysql.MySqlSourceReader.initializeEffectiveOffset(MySqlSourceReader.java:1075)
    at org.apache.doris.cdcclient.source.reader.mysql.MySqlSourceReader.generateMySqlConfig(MySqlSourceReader.java:960)
Caused by: java.sql.SQLException: Connections using insecure transport are prohibited while --require_secure_transport=ON.

That is exactly the path described in the issue: the early JDBC connection inside initializeEffectiveOffset() is opened before configFactory.jdbcProperties(...) is set, so it carries no SSL settings and the server rejects it.

Note that only 1 of the 3 methods fails on unpatched code. That is intended. The non-TLS control case runs against a second container without the secure-transport requirement, and the plaintext sanity check only asserts the server enforces TLS, so neither is affected by the ordering bug. A sibling test, MySqlStartupLatestITCase, passed in both runs, which rules out an environment problem.

Two notes on the test design, in case they come up in review:

1. require_secure_transport=ON is enabled after container startup via execInContainer, because the Testcontainers readiness probe connects without TLS and would otherwise never see the container as ready. For the same reason all SQL the test issues against that container goes through the mysql CLI over the Unix socket, which is exempt from the requirement, instead of JDBC.
2. The non-TLS control uses a separate container rather than toggling the flag off and on. With a shared container, a failure between the toggle and the restore would leave the requirement OFF and silently make the SSL assertions pass for the wrong reason.

I only covered offset=latest. earliest and the timestamp mode go through the same initializeEffectiveOffset() call, so they add no coverage of the ordering bug, and they would need different assertions since they replay the rows written during setup instead of skipping them. Happy to add them if you would rather have all three.

One thing worth flagging for CI: when I ran this locally the spring-boot-maven-plugin repackage goal moved the application classes under BOOT-INF/classes, and failsafe then failed with NoClassDefFoundError on org.apache.doris.cdcclient.common.Env before any test ran. I worked around it with -Dspring-boot.repackage.skip=true. If the project's integration-test job hits the same thing, that is the cause, and it is not specific to this test since the existing MySqlStartupLatestITCase failed identically.