-
Notifications
You must be signed in to change notification settings - Fork 76
Description
Pre-filing checklist
- I searched existing issues and didn't find a duplicate
Component(s)
Rust OTAP dataflow (rust/otap-dataflow/)
Bug Description
recv_when(false) only checks if the pdata channel is closed when the buffer is empty (is_empty() == true). If the upstream sends messages and then shuts down, the channel is closed but has buffered messages. is_empty() returns false, the closure check is skipped, and recv_when blocks on the control channel waiting for a Shutdown that never arrives - because the engine only sends Shutdown via control to receivers, not to processors or exporters.
This means any node calling recv_when(false) can hang if the pdata channel closes while there are still buffered messages.
Known callers affected:
- Azure Monitor exporter -
accepting_pdatais false during auth outages (see comment in azure monitor exporter: move auth to event loop only - non-blocking auth. #2312) - Fanout processor -
accept_pdata()returns false when inflight count hits max_inflight
Steps to Reproduce
- Have a node that calls
recv_when(false)for an extended period (e.g., Azure Monitor exporter during auth failure) - Upstream sends a few pdata messages
- Upstream shuts down (drops its pdata sender)
- The node's pdata channel is now closed with buffered messages
recv_when(false)never detects the closure - node hangs
Expected Behavior
recv_when(false) should detect pdata channel closure regardless of whether the buffer is empty, so the node can shut down cleanly.
Actual Behavior
recv_when(false) skips the closure check when the buffer is not empty, and the node hangs forever.
OTel-Arrow Version
latest
Environment
All env
Configuration
Log Output
Additional Context
No response
Metadata
Metadata
Assignees
Labels
Type
Projects
Status