fix: avoid buffering unbounded repartition output indefinitely

[goutamadwant]

Which issue does this PR close?

• Closes RepartitionExec withholds all output from an unbounded input until batch_size rows accumulate #24044.

Rationale for this change

RepartitionExec coalesces small batches on the producer side until the configured batch size is reached or all input senders finish. An unbounded input may never finish, so an available partial batch can be withheld indefinitely. This prevents an incremental query from emitting rows that are ready for downstream processing.

What changes are included in this PR?

• Skip producer-side batch coalescing when the input is unbounded.
• Preserve the existing coalescing behavior for bounded inputs and preserve-order execution.
• Add a regression test with an unbounded source that emits one partial batch and then remains open.

Are these changes tested?

Yes.

• cargo fmt --all -- --check
• cargo test -p datafusion-physical-plan
• cargo clippy --all-targets --all-features -- -D warnings
• The extended workspace test command required by AGENTS.md, including all 503 sqllogictest files

The regression test fails on the previous implementation because the partial batch is never emitted, and passes with this change. The existing bounded-input coalescing test also continues to pass.

Are there any user-facing changes?

Yes. Repartitioning an unbounded input can now emit partial batches promptly instead of waiting indefinitely to fill the configured batch size. This may expose smaller batches to downstream operators. There are no public API changes.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 89.74359% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.05%. Comparing base (eec8b94) to head (f267045).

Files with missing lines	Patch %	Lines
datafusion/physical-plan/src/repartition/mod.rs	89.74%	1 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #24193      +/-   ##
==========================================
- Coverage   81.05%   81.05%   -0.01%     
==========================================
  Files        1106     1106              
  Lines      382287   382325      +38     
  Branches   382287   382325      +38     
==========================================
+ Hits       309851   309877      +26     
- Misses      54121    54128       +7     
- Partials    18315    18320       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[calvinchengx]

Tested this against the repro from #24044, with a control on the parent commit so the result shows the fix flipping the behaviour rather than just passing on its own.

	no repartition	RoundRobin(4), batch_size 8192	RoundRobin(4), batch_size 1
eec8b947c (parent)	59	0	59
f26704525 (this PR)	59	59	59

Rows delivered in a six second window from a source emitting one row per 100ms, so 59 is the expected full count.

The middle column is the bug. The other two are unchanged across both commits: the first never had a repartition to coalesce, and the third already worked because batch_size=1 completes every batch immediately. So the change is scoped to the case it targets.

Method: a single clone of this repo with the repro added as datafusion-examples/examples/repartition_repro.rs, one git checkout between runs, rebuilt in place. I confirmed is_unbounded was absent from repartition/mod.rs on the parent rather than assuming the checkout took.

This resolves the issue as I reported it.