Pick a depth. Each prompt opens in your AI pre-loaded with the lesson. Click a row to preview the prompt.
By the time you're ready to build a real voice agent, you face a decision that shapes every module after this one: do you assemble your own pipeline from separate ASR, LLM, and TTS providers glued together with an orchestration framework (LiveKit, Pipecat), or do you use a native speech-to-speech model that handles the whole loop as one API call (OpenAI's Realtime API, Gemini Live)? Neither is strictly better — the assembled pipeline gives you provider choice, cost control, and the ability to swap any one component, but it's more moving parts to maintain. The single-model approach is dramatically simpler to integrate and often has lower latency because there's no serialization to text in between, but you're locked to whatever that model does well, and you lose fine-grained control over each stage. Most production teams end up with a considered answer to this question rather than a default, and this task gives you the framework to make that call instead of copying whatever the last blog post used.
This decision helper encodes the tradeoffs as a small function — not a real production decision-maker, but a scaffold for the questions you should actually be asking before picking a stack.
def recommend_stack(need_multiple_languages_per_call: bool,
need_fine_grained_tool_control: bool,
latency_is_the_top_priority: bool,
need_to_swap_providers_later: bool) -> str:
if need_to_swap_providers_later:
return "assembled pipeline (ASR+LLM+TTS) via Pipecat/LiveKit -- provider independence matters more than simplicity"
if latency_is_the_top_priority and not need_fine_grained_tool_control:
return "native speech-to-speech (OpenAI Realtime / Gemini Live) -- no text-serialization tax"
if need_fine_grained_tool_control:
return "assembled pipeline -- you need visibility into the text between stages to control tool-calling precisely"
if need_multiple_languages_per_call:
return "assembled pipeline -- pick best-in-class ASR per language, S2S models are less mature multilingually"
return "either -- start with the simplest (native S2S) and graduate to assembled if you hit a real limit"
print(recommend_stack(False, False, True, False)) # a fast FAQ voice bot
print(recommend_stack(True, False, False, False)) # multilingual support line
print(recommend_stack(False, True, False, False)) # complex tool-calling agent
print(recommend_stack(False, False, False, True)) # cost-sensitive, will shop providerspython3 main.py