[AMORO-3928] Modify the optimizer to support obtaining tasks from each AMS node for processing.

[wardlican]

Why are the changes needed?

In master-slave mode, the optimizer needs to support pulling optimization tasks from each ams node for processing.

Close #3928.

Brief change log

1. In master-slave mode, the optimizer can pull tasks from each currently active AMS.
2. In master-slave mode, the AMS automatically detects the addition and timeout of the optimizer from the database.

How was this patch tested?

• Add some test cases that check the changes thoroughly including negative and positive cases if possible

• Add screenshots for manual tests if appropriate

• Run test locally before making a pull request

Documentation

• Does this pull request introduce a new feature? (yes / no)
• If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

[github-actions]

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@amoro.apache.org list. Thank you for your contributions.

[wardlican]

Please help with the code review.

[czy006]

After resolving any conflicts in the code, I will initiate the first review on my end cc @wardlican

[czy006]

After resolving any conflicts in the code, I will initiate the first review on my end cc @wardlican

[wardlican]

After resolving any conflicts in the code, I will initiate the first review on my end cc @wardlican

Okay, I will resolve the conflicts based on the latest master branch and add support for a database-based storage solution for bucket allocation information, based on the latest HA (High Availability) scheme. @czy006

[codecov-commenter]

Codecov Report

❌ Patch coverage is 7.13202% with 612 lines in your changes missing coverage. Please review.
✅ Project coverage is 29.65%. Comparing base (cc03949) to head (70fda5d).
⚠️ Report is 6 commits behind head on master.

Files with missing lines	Patch %	Lines
...n-java/org/apache/amoro/api/OptimizingService.java	0.00%	277 Missing ⚠️
...ain/java/org/apache/amoro/client/AmsThriftUrl.java	0.00%	60 Missing ⚠️
...o/server/ha/DataBaseHighAvailabilityContainer.java	0.00%	48 Missing ⚠️
.../apache/amoro/optimizer/common/AmsNodeManager.java	0.00%	47 Missing ⚠️
...ro/optimizer/common/AbstractOptimizerOperator.java	13.46%	42 Missing and 3 partials ⚠️
...e/amoro/optimizer/common/ThriftAmsNodeManager.java	0.00%	43 Missing ⚠️
...ache/amoro/optimizer/common/OptimizerExecutor.java	55.17%	20 Missing and 6 partials ⚠️
...a/org/apache/amoro/server/ZkBucketAssignStore.java	0.00%	20 Missing ⚠️
.../apache/amoro/server/DefaultOptimizingService.java	22.22%	14 Missing ⚠️
...a/org/apache/amoro/server/DBBucketAssignStore.java	0.00%	12 Missing ⚠️
... and 7 more

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3950      +/-   ##
============================================
- Coverage     29.90%   29.65%   -0.26%     
- Complexity     4160     4235      +75     
============================================
  Files           670      677       +7     
  Lines         53533    54594    +1061     
  Branches       6797     6942     +145     
============================================
+ Hits          16008    16188     +180     
- Misses        36343    37205     +862     
- Partials       1182     1201      +19     

Flag	Coverage Δ
core	29.65% <7.13%> (-0.26%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@amoro.apache.org list. Thank you for your contributions.

[github-actions]

This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

[xxubai]

Things are moving quickly. I’ve left some comments for your reference.

[czy006]

LGTM.After the overall functionality is completed, we need to conduct a comprehensive test