Turn-taking, at a glance

Why this matters

Before this course spends two full modules on the mechanics of turn-taking, you need the one-paragraph version so the rest of the stack makes sense around it. A voice agent has to constantly decide three things in real time: is the user still talking, has the user finished their turn, and is it safe for the agent to start talking without talking over someone. Text chat never has to answer any of these questions — there's always an explicit 'send' button. Voice has no send button, which is why voice agents need Voice Activity Detection (VAD) to sense presence of speech, endpointing logic to decide a turn is over, and barge-in handling to let the user interrupt. Getting this wrong is the single most common reason a voice demo feels 'off' even when the ASR, LLM, and TTS are each individually excellent — the seams between them are where quality actually lives.

Demo

A minimal turn-taking state machine below models three states — listening, thinking, speaking — and the events that move between them, without any real audio yet. You'll wire real VAD in Module 5; this is the skeleton it will plug into.

Try it yourself

1. Run it and find the line where user_barge_in fires mid-SPEAKING — confirm the state snaps back to LISTENING instead of finishing the reply.
2. Feed on_event an event the state machine doesn't handle in the current state (e.g. reply_ready while LISTENING) and observe that the state silently doesn't change — then add an else branch that raises instead.
3. Add a fourth state, INTERRUPTED, that SPEAKING moves to on user_barge_in, before returning to LISTENING on a new endpoint_detected.
4. Add a counter that increments every time user_barge_in fires, and print the total barge-in count for the session.

References

• Pipecat docs — turn detection
• LiveKit docs — end-of-turn detection

Chat about this lesson