test(e2e): durable rustisk trunk e2e suite + M2 harness race fix#166
Merged
Conversation
added 2 commits
July 18, 2026 15:52
Add tests/e2e-trunk/: a self-contained, CI-runnable end-to-end proof of the inbound-trunk PIN-gate path, turning the one-shot M9 arming harness into a committed, re-runnable suite. - Hermetic: synthetic Chime UAC (chime_caller.py) -> rustisk PinGate -> GRANTED -> Dial(PJSIP/qa-bridge) -> a MOCK registered SIP endpoint (mock_bridge.py: inbound REGISTER + MD5 digest + answer + distinct-tone media) -> two-way RTP, each direction proven by a distinct tone detected on the FAR side (caller 440 Hz, mock 660 Hz). Wrong PIN -> REJECTED -> the bridge is never INVITEd (invite_count==0). No FreeSWITCH/Mumble/docker/ NET_RAW/root -- only the rustisk binary + python3 stdlib (Goertzel, no numpy) + loopback. Runs on fresh 127.0.0.6x / 35xxx-36xxx ports, preflight-asserted free and non-overlapping with the live trunk / m0 / m9 rigs. - Wired into CI as the e2e-trunk job (build rustisk + run.sh on ubuntu-latest). - integration/: the original M9 full-bridge validators (chime_caller.py + qa_probe.py, numpy) + sanitized live config (qa-bridge password redacted). Requires the real Mumble + pymumble bridge voice stack; NOT run in CI -- it is the operator-run live-cutover validator. - Fix the M2 sniffer-startup race in tests/freeswitch-pin-gate: rtp_injector.py gains a --go-file priming gate (keep reading/classifying frames but don't start the match deadline until the caller confirms the flow-triggering ESL round trip returned), and run.sh touches the go-file only after fs_cli returns -- so docker-exec/ESL setup latency no longer eats the 5s window. Adds an m2-standalone case selector.
- Raise MIN_TONE_RATIO 20 -> 5000. At 20 it sat BELOW the ~150-360 Goertzel noise floor and never discriminated signal from noise; also require the target band to beat every other band by 10x (not merely be the max). Closes the narrow window where a silence/comfort-noise partial relay could pass on a coincidental band "win". - Assert the received RTP source IP == rustisk's media IP on BOTH legs, so the suite proves rustisk RELAYED the audio (defends against a direct_media/ peer-to-peer regression), not merely that media crossed end-to-end. - mock_bridge: wake tx_loop's answer-wait on shutdown so the REJECTED path never parks a thread; extend the post-stop drain to 0.3s (> the 0.25s rx socket timeout) so rx_loop settles before the samples are read.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Turns the proven-but-one-shot M9 trunk arming harness into a committed, re-runnable, hermetic e2e suite for rustisk's inbound-trunk PIN-gate path, and folds in the M2 sniffer-startup race fix.
tests/e2e-trunk/— self-contained, CI-runnable e2e (the key deliverable)Proves the full trunk path with no external services (no FreeSWITCH, Mumble, docker, NET_RAW, or root — only the rustisk binary +
python3stdlib + loopback):The pymumble bridge is replaced by
mock_bridge.py: a tiny UAS that REGISTERs to rustisk with MD5 digest auth (realmasterisk, qop=auth), answers rustisk'sDial, and runs bidirectional media. Receiver-side assertions (like M0/M2):Verbose: PIN_GATE_RESULT=REJECTEDand the mock'sinvite_count == 0(the reject never reachedDial).Verbose: PIN_GATE_RESULT=GRANTED; A→bridge proven by the mock receiving the caller's relayed 440 Hz tone (invite_count==1,rx_voice≥100, 440 Hz dominant); bridge→A proven by the caller receiving the mock's distinct 660 Hz tone (voice_rx≥100, 660 Hz dominant).Distinct tones per direction (not an echo) make each RTP direction independently attributable by frequency. Tone detection is a stdlib Goertzel filter, so CI needs no numpy.
Isolation: fresh loopback IPs + high ports (rustisk
127.0.0.60:35060, caller127.0.0.61, mock127.0.0.62, RTP36000-36100, AMI35038), preflight-asserted free and explicitly non-overlapping with the live trunk (:45070), the M9 arming rig (:25060/:25038/:25062/:21000-21100) and M0 (:15060/:15038).Proven green — run locally 4×, stable (rustisk @ 461ad74)
The expected tone dominates the next-strongest band by ~4 orders of magnitude in both directions on every run. Frame counts + tone dominance are the load-bearing assertions.
tests/e2e-trunk/integration/— full-stack M9 validator (NOT in CI)The original M9
chime_caller.py+qa_probe.py(numpy) + sanitized live config (qa-bridge auth password redacted to a placeholder; real PIN staged at runtime). Requires the real qa-sip Mumble channel + pymumble bridge — operator-run live-cutover validator, not a CI gate.M2 harness race fix (
tests/freeswitch-pin-gate/)rtp_injector.pygains a--go-filepriming gate: it keeps reading/classifying frames but does not start the 5s match deadline until the caller confirms (via the go-file) that the flow-triggeringfs_cli/ESL round trip returned — so docker-exec/ESL setup latency no longer gets deducted from the match window (the documented intermittent "no matching RTP packet observed" flake).run.shtouches the go-file only afterfs_clireturns, and gains anm2-standalonecase selector.CI
Adds an
e2e-trunkjob to.github/workflows/ci.yml(buildrustisk+ runtests/e2e-trunk/run.shonubuntu-latest). The suite is hermetic (loopback + python3 stdlib), so it runs unattended with no runner privileges. The docker-basedtests/freeswitch-pin-gateharness is not CI-wired (needs FreeSWITCH/docker) — the M2 fix ships but is exercised manually, consistent with the other shell harnesses.Test plan
tests/e2e-trunk/run.sh— 4/4 green locally (2 required + 2 extra for stability).python3 -m py_compile,bash -n,shellcheck -S errorall clean.