[py][bidi] make the WebSocket transport thread-safe and event-driven#17682
[py][bidi] make the WebSocket transport thread-safe and event-driven#17682AutomatedTester wants to merge 2 commits into
Conversation
Replace the busy-wait response loop with per-request threading.Event waits, lock the shared message/callback state, and dispatch events on a single long-lived thread instead of spawning one thread per event. This removes the CPU burn and latency floor on every command, bounds thread usage so high event volume can no longer exhaust threads, preserves event ordering, and surfaces callback exceptions instead of losing them on an orphaned thread. close() now stops the dispatcher, clears handlers, and wakes any caller still blocked on a response so it fails fast. Wire behaviour is unchanged. Adds unit tests covering concurrent execute() response routing, single-threaded dispatch, callback error isolation, and teardown.
The thread-safe transport no longer busy-waits, so command responses return the instant they arrive instead of up to one poll interval later, and events are delivered through a single dispatcher thread. Events and rendering effects are therefore no longer guaranteed to have landed by the time the triggering command returns -- which is the correct, documented contract. Update the event and wheel-scroll tests to await their outcome instead of asserting immediately: - browsing_context event tests wait for the event to arrive; context_created cases assert on the specific context rather than event counts, since that event may report more than one context per creation. - removal tests assert the post-removal context is never observed instead of comparing counts. - wheel tests wait for the scroll position to settle before asserting.
|
Follow-up ( Why they failed: the old transport busy-waited (so every command returned up to one poll interval after its response actually arrived) and dispatched each event on its own thread immediately. Tests relied on that incidental slack to assert event/DOM state on the line right after the triggering command. With the new transport, responses return instantly and events are delivered via the dispatcher thread, so they are no longer guaranteed to have landed by the time the command returns. Contract going forward (documented here intentionally): BiDi events are delivered asynchronously — do not assume a handler has run by the time the triggering command returns; await the event. This matches the direction of the Test changes (no production behavior change):
Verified locally on |
2 participants
Description
Foundation for safe parallel use of the Python BiDi API (driving multiple browsing contexts/tabs concurrently over one connection). This is the P0 "transport correctness & efficiency" step: make the single WebSocket safe and efficient under concurrent use without changing wire behaviour.
What changed in
websocket_connection.pyexecute()registers a per-requestthreading.Eventbefore sending and blocks onevent.wait(timeout); the receive thread sets it when the matching response arrives. Removes the CPU burn and theinterval-sized latency floor on every command. The connection-open handshake uses the same pattern instead of polling._messages+ per-request waiters under one lock;callbacksunder another. No longer relying on the GIL for correctness.ERRORwith traceback, and keeps delivering. Command failures still raise synchronously inexecute()(fail-fast preserved).close()stops the dispatcher, clears handlers, and wakes any caller blocked on a response so it fails fast instead of waiting out the full timeout.The
(url, timeout, interval)constructor signature is unchanged (ABI);intervalis now a no-op but retained.Tests
New
test/unit/selenium/webdriver/common/websocket_connection_tests.py(7 tests,small): single-response routing, 25-way concurrentexecute()with responses fed back in reverse order (proves per-caller routing under contention), timeout, single-thread dispatch, callback-exception isolation + logging, handler clearing on close, and close waking blocked callers. The tests override only the network boundary (_start_ws); all transport logic runs for real.Motivation and Context
Driving N tabs concurrently today means N threads making blocking busy-wait calls against unlocked dicts, with an unbounded thread spawned per event. This PR fixes the mechanical/transport side so in-process threading over one driver's contexts is safe and efficient. It's the enabling work for the per-context handle object and event-scoping work that follow.
Types of changes