Voice agent platforms: an architecture fit assessment

1. Executive summary

The honest headline: for most enterprises the framework is not the hard part, and the decision is really build-vs-buy wearing a technical costume. LiveKit and Pipecat are both good. Either will work. What separates them is what you already have and what you're willing to operate — and the far more consequential choice is whether you run the pipeline at all or rent it from Vapi and accept a per-minute markup in exchange for not having an on-call rotation for a media server.

My default for a voice-core product where margin matters at scale is LiveKit, and the reason is telephony: LiveKit shipped native SIP and phone numbers in 2025, so inbound and outbound calling no longer needs a Twilio bridge sitting in the middle of your latency budget. For a team whose product is not voice — voice is one channel among several — I'd start on Vapi and revisit at the point where the per-minute bill exceeds the cost of an engineer.

Verdict: Trial (LiveKit, Pipecat) / Narrow Use (Vapi, managed clouds)
Best fit: LiveKit  — voice-core products, WebRTC + SIP, margin at scale
          Pipecat  — custom/existing media infra, maximum pipeline control
          Vapi     — speed to market, voice as a secondary channel
          Managed  — deep single-cloud shops with hard residency constraints
Main risks: turn-taking quality is yours to own on every option; per-minute
          economics degrade badly on long calls; managed clouds trade model
          flexibility for compliance paperwork you may not need.
Required mitigations: budget latency stage-by-stage before choosing; build the
          eval harness before the agent; keep the LLM behind a gateway so the
          middle of the pipeline stays swappable.
Recommended next step: 4-week POC on YOUR call recordings and YOUR tool
          latency — not the vendor's demo.

Best-fit workloads: inbound support deflection, outbound reminders and confirmations, appointment booking, qualification and routing. Poor-fit workloads: anything where a wrong answer is unrecoverable and there's no human escalation path; emotionally sensitive conversations; long unstructured calls where the per-turn cost curve gets ugly. Top strengths of the category: the frameworks have absorbed the genuinely hard media engineering. Top risk: every one of these platforms gives you a working demo in an afternoon and none of them gives you good turn-taking for free.

2. Platform positioning

The four options are not the same kind of thing, which is the first thing to get straight.

LiveKit is infrastructure-first. Its core is an open-source Go SFU, and its organizing abstraction is the room: an agent joins as a headless participant alongside human participants. That means multi-party scenarios — a supervisor listening in, a warm transfer to a human, two callers plus an agent — are native rather than bolted on. Pipecat (Python, from Daily) is pipeline-first: a frame-based streaming model where you compose VAD → STT → LLM → TTS processors explicitly, with interruption handling built into the frame protocol. Vapi is a managed product: the pipeline is someone else's problem and you configure it. The managed cloud stacks (Bedrock with Nova Sonic, Vertex's voice agent surface) are the single-cloud answer — your data never leaves the boundary your compliance team already signed off on.

Both open frameworks published semantic turn-detection models, and the different bets are informative. Pipecat's Smart Turn v2 is acoustic-first: a wav2vec2 encoder with a linear head that reads the raw waveform and emits one completion probability, no transcript involved, multilingual across 14 languages. LiveKit's turn detector is a fine-tune of Qwen2.5-0.5B-Instruct — a small language model, chosen for CPU-latency reasons — that combines semantic understanding with acoustic cues like intonation and rhythm (LiveKit engineering blog). Acoustic-first is faster and more portable; semantic-plus-acoustic is better on the sentences where the words carry the signal.

That split isn't just a product decision — it mirrors an open research question. The Easy Turn work on full-duplex spoken dialogue argues for fusing acoustic and linguistic modalities precisely because neither alone is robust (arXiv:2509.23938). Read that way, LiveKit and Pipecat are running the two arms of the same experiment in production, and the honest state of the art is that nobody has this solved. I dig into the practical consequences in the turn-taking article.

3. Radar criteria — the reasoning

Latency & turn-taking

All four can hit a respectable time-to-first-audio; the spread is in how much control you have when it isn't. LiveKit and Pipecat expose every stage boundary, so you can overlap, prefill, and tune endpointing per workflow step. LiveKit 4.6, Pipecat 4.5 — LiveKit edges it on the telephony path having one less hop. Vapi is fast out of the box and hard to tune past its defaults: 4.0. The managed clouds are fast within their own model family: 4.2.

Telephony & channels

This is the axis that decides most enterprise builds, because the phone is where the calls are. LiveKit's native SIP and phone numbers removed the Twilio bridge — 4.6. Vapi has telephony as a first-class product feature: 4.4. Pipecat gets PSTN/SIP through Pipecat Cloud, solid but younger: 3.9. The managed clouds generally expect you to bring a carrier: 3.4.

Tool calling & orchestration

Pipecat's explicit pipeline makes it the easiest to reason about when a tool call needs to block, run async, or emit a filler phrase while it waits — 4.3. LiveKit 4.2. Managed clouds 4.0, constrained to their own function-calling story. Vapi 3.9: fine for straightforward calls, awkward when orchestration gets genuinely stateful — at which point you're building an agent harness inside a config file, which is a bad place to build one.

Model flexibility

Pipecat is the most pluggable of the four by design — 4.7. LiveKit 4.5. Vapi restricts you to a supported vendor menu: 3.5. Managed clouds steer hard toward first-party models; that's the trade you're making: 2.8.

Scale & elasticity

Concurrency is per-call-stateful, which makes voice scaling unlike request-response scaling — every call pins media resources for its whole duration. Managed clouds 4.6 and Vapi 4.2 hand you this. LiveKit 4.4 (the SFU is proven at scale, and LiveKit Cloud removes the work). Pipecat self-hosted needs real capacity engineering, though Pipecat Cloud advertises autoscaling with unlimited concurrency: blended 3.9.

Reliability / HA / DR

Managed clouds 4.6. LiveKit 4.2 — the SFU has years of production WebRTC behind it. Vapi 4.0. Pipecat 3.8, mostly a function of self-hosted deployments being as reliable as the team running them. Assumption, flagged: I'm scoring the open frameworks on a competent-but-not-specialist platform team.

Governance & compliance

Managed clouds win this one outright — 4.7 — because "the audio never left our AWS account" ends a conversation that otherwise takes six weeks. Pipecat Cloud advertises HIPAA/GDPR compliance: 4.0. LiveKit 3.9 (self-hosting gives you full data control, at the cost of owning the controls). Vapi 3.6: fine for general commercial use, more friction in regulated deployments where call audio contains PII by definition — see PII handling in analytics for why "we'll redact it later" is not a plan.

Observability & evals

Weak across the board, which is the category's real gap. Vapi 4.1 has the best built-in call review of the four. Managed clouds 4.0 inherit their cloud's tracing. LiveKit 3.8, Pipecat 3.7 — both emit good traces and neither gives you a voice-specific eval harness. Everyone is building this themselves right now, and roughly 42% of production voice issues are voice-specific — invisible to transcript-only evaluation (Hamming, voice agent testing guide). The academic side is only now catching up: EVA-Bench proposes an end-to-end evaluation framework for voice agents precisely because text-level benchmarks miss the failures that matter (arXiv:2605.13841). Budget for this regardless of platform — it is the axis where your own harness matters more than the vendor's.

Ecosystem & integration

LiveKit 4.3 — the WebRTC ecosystem is deep and the SDK coverage is broad. Vapi 3.8, Pipecat 4.0 (Python means every AI library is already there). Managed clouds 3.9, excellent inside their boundary and indifferent outside it.

TCO & cost control

A mainstream cascaded stack runs roughly $0.08–$0.15 per conversation-minute plus about $0.013/min per telephony leg. The open frameworks let you pay that cost directly: Pipecat 4.2, LiveKit 4.0. Vapi adds platform margin on top of the same underlying vendors, which is entirely fair and entirely real: 2.9. Managed clouds 3.4 — bundled, predictable, rarely cheapest. The thing that dominates all of this at scale isn't the platform: it's that on a 30-minute call the LLM share of cost climbs to roughly 47% because history is re-sent every turn.

4. Radar scorecard

How these scores are arrived at — the rings, the 0–5 scale, and the evidence rules — is written up in the radar methodology. The numbers on this page — this table and the interactive chart — come from one file: scorecards.json. It is the source of truth, it is machine-readable, and keeping it separate is what stops three copies of the same scores drifting apart between editions. Last reviewed 2026-Q3.

Comparator scores (same axis order) — Pipecat: 4.5 / 3.9 / 4.3 / 4.7 / 3.9 / 3.8 / 4.0 / 3.7 / 4.0 / 4.2. Vapi: 4.0 / 4.4 / 3.9 / 3.5 / 4.2 / 4.0 / 3.6 / 4.1 / 3.8 / 2.9. Managed cloud: 4.2 / 3.4 / 4.0 / 2.8 / 4.6 / 4.6 / 4.7 / 4.0 / 3.9 / 3.4.

5. The radar

Ten-axis fit for the enterprise phone-agent use case. Higher is lower-risk for this workload, not "better in the abstract" — the managed-cloud shape is exactly what you'd want if compliance is your binding constraint and model choice isn't. Click any name in the legend to toggle it on or off — comparing two shapes at a time is far more legible than comparing four.

Read the shapes rather than the totals. LiveKit and Pipecat are nearly the same polygon rotated slightly: LiveKit bulges toward telephony and scale, Pipecat toward model flexibility and cost. Vapi's dent is entirely on the cost axis, and that dent is the product — you're buying time. The managed clouds are the odd shape: excellent everywhere the compliance team looks, weakest exactly where the engineers look.

6. Workload-specific analysis

7. Reference architecture

The pattern I'd build for the target use case: carrier into SIP, the agent as a participant in a room, the LLM behind a gateway so the middle of the pipeline stays swappable, tools exposed over MCP where the boundary is genuinely external, and every call recorded into an eval loop from day one rather than after the first bad quarter.

graph LR
  P["PSTN / carrier"] --> SI["SIP trunk"]
  W["Web / mobile SDK"] --> RT["WebRTC"]
  SI --> RM["Media room
(SFU)"]
  RT --> RM
  RM --> AG["Agent worker
VAD + turn detector"]
  AG --> ST["STT"]
  ST --> GW["LLM gateway
routing + fallback"]
  GW --> LM["LLM"]
  LM --> TT["TTS"]
  TT --> RM
  LM -.->|"tool calls"| MC["MCP /
internal APIs"]
  MC -.-> CR[("CRM / booking
/ knowledge base")]
  RM --> RC["Recording
+ transcript"]
  RC --> EV["Eval harness
golden + replay"]
  RC --> AS[("Audit store")]
  PI["PII redaction"] -.->|"governs"| AS

The configuration detail that matters most, and the one I'd put in the first commit — endpointing patience varying by what you just asked:

# Turn-detection patience by workflow step, not one global threshold.
# A pause after "what's your account number?" is almost certainly mid-utterance.
turn_detection:
  default:
    completion_threshold: 0.55     # semantic turn model probability
    max_silence_ms: 700
  steps:
    collect_account_number:
      completion_threshold: 0.80   # demand strong evidence the turn ended
      max_silence_ms: 2000         # people chunk digits
    collect_address:
      completion_threshold: 0.75
      max_silence_ms: 1800
    confirm_yes_no:
      completion_threshold: 0.45   # a short pause here IS the answer
      max_silence_ms: 400

barge_in:
  backchannel_phrases: ["mhm", "uh huh", "yeah", "right", "okay"]
  min_speech_ms: 220               # a click or cough must not stop playback
  truncate_context_on_interrupt: true   # model history = what was HEARD

8. POC plan (4 weeks)

• Week 1 — instrument before you build. Stand up the eval harness first: 20 golden conversations drawn from real call recordings, scored on task completion, P95 dead air, and clipped words. If you can't measure turn-taking on day one you will not measure it at all.
• Week 2 — one flow, two platforms. Build the single highest-volume call flow on LiveKit and on Vapi. Same STT, same LLM, same voice. You're comparing the pipeline, not the vendors underneath it.
• Week 3 — break it on purpose. Real PSTN, not WebRTC. Accented speech, background noise, someone reading a 16-digit number, an interruption mid-sentence, and a tool call you've deliberately slowed to 2 seconds. This is the week the answer usually becomes obvious.
• Week 4 — decide on numbers. Re-score this table with your measured latency, your missed-barge-in rate, and your actual cost per minute at your median call length. Write the ADR. Commit to Adopt / Narrow-Use / Defer.

9. Final recommendation

Trial LiveKit if voice is your product; use Vapi for everything else. That's the short version, and for most teams it's the whole decision.

LiveKit gets the nod because its native SIP support removes a hop from the latency budget on the channel that actually carries your calls. Every other advantage in this category is a rounding error next to the phone working well. Pipecat is the better pick if you already run media infrastructure or need maximum model pluggability — the gap to LiveKit is genuinely small, and the choice comes down to what your team already knows rather than to any technical verdict I can hand you.

Vapi's per-minute markup is cheaper than an engineer, right up until it isn't. Do that arithmetic at your projected volume rather than today's, because the crossover arrives faster than anyone plans for. And the managed clouds win exactly one argument: data residency. If that's a binding constraint for you it's a real reason, and the flexibility you give up is worth paying.

Three things hold regardless of which one you pick. Build the eval harness before the agent — voice failures are inaudible in a transcript and invisible in a latency dashboard, so an agent with no harness is an agent whose quality you are guessing at. Keep the LLM behind a gateway, because the middle of the pipeline will change and you want that to be a config edit rather than a migration. And model your cost at p90 call length, not median; the tail is where the economics actually live.

I'd re-open this assessment when speech-to-speech tool calling matures. That single change would redraw the whole radar — it collapses the cascade's main advantage — and it's closer than it was a year ago.

References

Every score above traces back to something. Primary sources and docs first, then the research where the underlying problem is still open — the turn-taking and evaluation axes in particular rest on questions the literature has not closed.

Scores are my own architect-level judgment for a specific use case, calibrated to mid-2026 releases — not a vendor ranking. The axes differ from my analytics assessments, so scores are not comparable across pages. Re-score against your own POC numbers before you commit.