Repository navigation
fix(avatar): restore previous audio output on AvatarSession.aclose() - #7477
dajiaohuang wants to merge 2 commits into
Conversation
When an AvatarSession installs its DataStreamAudioOutput via replace_audio_tail(), it never restored the previous audio tail when aclose() ran. If avatar.start() failed (or the session was closed early), the agent remained muted because subsequent audio went to the orphaned avatar DataStreamAudioOutput. - Add _replace_audio_tail() helper that saves the prior audio tail - Restore the saved tail in aclose() before other cleanup - Update all avatar plugins (inference, anam, keyframe, bey, trugen, synthesia, lemonslice, runway, tavus, protoface, avatario, avatartalk, bithuman, did, liveavatar, simli, spatius) to use the helper Fixes livekit#7276
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
Devin Review found 3 potential issues.
2 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)
| if self._agent_session and self._previous_audio_output is not None: | ||
| try: | ||
| self._agent_session.output.replace_audio_tail(self._previous_audio_output) |
There was a problem hiding this comment.
🔴 Audio remains routed to closed avatar
When an avatar starts before the agent has an audio output, _replace_audio_tail saves None. Later, aclose() skips restoration, leaving speech routed to the closed avatar.
Learn more
A common startup order runs AvatarSession.start() before AgentSession.start(). In that order AgentOutput.audio can be None when the avatar sink is installed, so _previous_audio_output remains None. The agent's startup then disables its RoomIO audio output because an avatar sink is already configured AgentSession.start. The new close guard skips restoration for None; subsequent speech still targets the orphaned avatar sink.
Example: A fresh agent has no output.audio. Start a LemonSlice avatar, start the agent, then close the avatar while the agent continues running. The agent's output remains the avatar data stream, not a working room audio output.
Recommended fix: Track whether an avatar sink was installed separately from whether a previous sink existed. Define a valid fallback for the no-prior-sink case, including the agent's RoomIO output lifecycle when the avatar preceded AgentSession.start().
Was this helpful? React with 👍 or 👎 to provide feedback.
| if self._agent_session and self._previous_audio_output is not None: | ||
| try: | ||
| self._agent_session.output.replace_audio_tail(self._previous_audio_output) |
There was a problem hiding this comment.
🔴 Older avatar close overrides newer audio
If another avatar replaces the tail, the earlier avatar's aclose() restores its stale output over the new avatar. The newer avatar stops receiving speech; closing it can reinstall the earlier avatar's orphaned sink.
Learn more
Each avatar caches its predecessor independently, but close never checks whether that avatar still owns the current tail. Two avatars can be started on the same agent without a guard in AvatarSession.start. Closing them out of start order swaps in an older sink and prevents the still-running avatar from receiving speech.
Example: Agent output is room; avatar A starts and saves room; avatar B starts and saves A's output. A closes and installs room over B. B closes and reinstalls A's now-orphaned output.
Recommended fix: Track the sink installed by each avatar and restore only when it remains the current tail; handle overlapping avatars with explicit ownership or a stack. Do not overwrite a tail installed after this avatar.
Was this helpful? React with 👍 or 👎 to provide feedback.
|
Thanks for the review. I checked the three cases against this head: it saves |
- Use getattr for defensive attribute access (works with test mocks) - Traverse next_in_chain to find _AudioSinkProxy and save its downstream sink - Import _AudioSinkProxy from io module
|
Fixed the unit test failures. The issue was in
Changes:
All 30 inference avatar tests and 166 synthesia plugin tests pass locally. |
This branch has not been deployed
2 participants
When an AvatarSession installs its DataStreamAudioOutput via replace_audio_tail(), it never restored the previous audio tail when aclose() ran. If avatar.start() failed (or the session was closed early), the agent remained muted because subsequent audio went to the orphaned avatar DataStreamAudioOutput.
Fixes #7276