fix(voice): report played audio across an audio sink swap - #6967
Open
longcw wants to merge 1 commit into
Open
Conversation
Replacing the audio tail mid-segment left the recording wrong three ways. A sink attached part-way through counts from its own zero, so its reports landed at the start of the segment. A detached sink's playhead was never heard, since the proxy stopped listening before clearing it. And a sink that reports no runs of its own, such as a remote avatar worker, lost its whole stretch once any other sink had reported. Room I/O now reports its playhead from clear_buffer(), which a detaching sink reaches when the playout wait does not. The proxy keeps listening for progress across that clear, rebases each report onto the segment, and reports for a sink that says nothing: all it was given counts as played, the same assumption made when a segment ends on it. playback_started and playback_finished stay unsubscribed before the clear. They decide when a segment ends, which is the proxy's call to make, and a clear can finish a segment that is still running. Detaching a remote sink after a flush still records its whole segment as played, which is all that is knowable once it is gone.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
replace_audio_tailcan replace the audio sink while a segment still plays.Each sink counts its playback from the first frame it received, not from the start of the segment.
The recorder therefore put the audio of a new sink at the start of the segment, and it dropped what a detached sink played.
A sink that reports no runs of its own, such as a remote avatar worker, lost all of its audio once another sink reported.
Fix
Room I/O now reports its playhead when it clears its buffer, because a sink that the proxy detaches never reaches the playout wait.
The proxy also hears progress across the clear, and it moves each report onto the segment timeline.
For a sink that says nothing, the proxy reports on its behalf, and everything the sink received counts as played.
playback_startedandplayback_finishedstill go quiet before the clear, because they decide when a segment ends and that call belongs to the proxy.When the proxy detaches a remote sink after a flush, the recording still counts its whole segment as played. That is all that is knowable once the sink is gone.
follow up of #6921 (comment)