Voice agent platforms: an architecture fit assessment

1. Executive summary

The honest headline: for most enterprises the framework is not the hard part, and the decision is really build-vs-buy wearing a technical costume. LiveKit and Pipecat are both good. Either will work. What separates them is what you already have and what you're willing to operate — and the far more consequential choice is whether you run the pipeline at all or rent it from Vapi and accept a per-minute markup in exchange for not having an on-call rotation for a media server.

My default for a voice-core product where margin matters at scale is LiveKit, and the reason is telephony: LiveKit shipped native SIP and phone numbers in 2025, so inbound and outbound calling no longer needs a Twilio bridge sitting in the middle of your latency budget. For a team whose product is not voice — voice is one channel among several — I'd start on Vapi and revisit at the point where the per-minute bill exceeds the cost of an engineer.

Verdict: Trial (LiveKit, Pipecat) / Narrow Use (Vapi, managed clouds)
Best fit: LiveKit  — voice-core products, WebRTC + SIP, margin at scale
          Pipecat  — custom/existing media infra, maximum pipeline control
          Vapi     — speed to market, voice as a secondary channel
          Managed  — deep single-cloud shops with hard residency constraints
Main risks: turn-taking quality is yours to own on every option; per-minute
          economics degrade badly on long calls; managed clouds trade model
          flexibility for compliance paperwork you may not need.
Required mitigations: budget latency stage-by-stage before choosing; build the
          eval harness before the agent; keep the LLM behind a gateway so the
          middle of the pipeline stays swappable.
Recommended next step: 4-week POC on YOUR call recordings and YOUR tool
          latency — not the vendor's demo.

Best-fit workloads: inbound support deflection, outbound reminders and confirmations, appointment booking, qualification and routing. Poor-fit workloads: anything where a wrong answer is unrecoverable and there's no human escalation path; emotionally sensitive conversations; long unstructured calls where the per-turn cost curve gets ugly. Top strengths of the category: the frameworks have absorbed the genuinely hard media engineering. Top risk: every one of these platforms gives you a working demo in an afternoon and none of them gives you good turn-taking for free.

2. Platform positioning

The four options are not the same kind of thing, which is the first thing to get straight.

LiveKit is infrastructure-first. Its core is an open-source Go SFU, and its organizing abstraction is the room: an agent joins as a headless participant alongside human participants. That means multi-party scenarios — a supervisor listening in, a warm transfer to a human, two callers plus an agent — are native rather than bolted on. Pipecat (Python, from Daily) is pipeline-first: a frame-based streaming model where you compose VAD → STT → LLM → TTS processors explicitly, with interruption handling built into the frame protocol. Vapi is a managed product: the pipeline is someone else's problem and you configure it. The managed cloud stacks (Bedrock with Nova Sonic, Vertex's voice agent surface) are the single-cloud answer — your data never leaves the boundary your compliance team already signed off on.

Both open frameworks published semantic turn-detection models, and the different bets are informative. Pipecat's Smart Turn is acoustic-first: a wav2vec2 encoder with a linear head that reads the raw waveform and emits one completion probability, no transcript involved, multilingual across 14 languages in v2. LiveKit's turn detector is a fine-tune of Qwen2.5-0.5B-Instruct — a small language model, chosen for CPU-latency reasons — that combines semantic understanding with acoustic cues like intonation and rhythm. Acoustic-first is faster and more portable; semantic-plus-acoustic is better on the sentences where the words carry the signal. I dig into why this matters in the turn-taking article; for the assessment, the relevant fact is that both ship something real and neither is a solved problem.

3. Radar criteria — the reasoning

Latency & turn-taking

All four can hit a respectable time-to-first-audio; the spread is in how much control you have when it isn't. LiveKit and Pipecat expose every stage boundary, so you can overlap, prefill, and tune endpointing per workflow step. LiveKit 4.6, Pipecat 4.5 — LiveKit edges it on the telephony path having one less hop. Vapi is fast out of the box and hard to tune past its defaults: 4.0. The managed clouds are fast within their own model family: 4.2.

Telephony & channels

This is the axis that decides most enterprise builds, because the phone is where the calls are. LiveKit's native SIP and phone numbers removed the Twilio bridge — 4.6. Vapi has telephony as a first-class product feature: 4.4. Pipecat gets PSTN/SIP through Pipecat Cloud, solid but younger: 3.9. The managed clouds generally expect you to bring a carrier: 3.4.

Tool calling & orchestration

Pipecat's explicit pipeline makes it the easiest to reason about when a tool call needs to block, run async, or emit a filler phrase while it waits — 4.3. LiveKit 4.2. Managed clouds 4.0, constrained to their own function-calling story. Vapi 3.9: fine for straightforward calls, awkward when orchestration gets genuinely stateful — at which point you're building an agent harness inside a config file, which is a bad place to build one.

Model flexibility

Pipecat is the most pluggable of the four by design — 4.7. LiveKit 4.5. Vapi restricts you to a supported vendor menu: 3.5. Managed clouds steer hard toward first-party models; that's the trade you're making: 2.8.

Scale & elasticity

Concurrency is per-call-stateful, which makes voice scaling unlike request-response scaling — every call pins media resources for its whole duration. Managed clouds 4.6 and Vapi 4.2 hand you this. LiveKit 4.4 (the SFU is proven at scale, and LiveKit Cloud removes the work). Pipecat self-hosted needs real capacity engineering, though Pipecat Cloud advertises autoscaling with unlimited concurrency: blended 3.9.

Reliability / HA / DR

Managed clouds 4.6. LiveKit 4.2 — the SFU has years of production WebRTC behind it. Vapi 4.0. Pipecat 3.8, mostly a function of self-hosted deployments being as reliable as the team running them. Assumption, flagged: I'm scoring the open frameworks on a competent-but-not-specialist platform team.

Governance & compliance

Managed clouds win this one outright — 4.7 — because "the audio never left our AWS account" ends a conversation that otherwise takes six weeks. Pipecat Cloud advertises HIPAA/GDPR compliance: 4.0. LiveKit 3.9 (self-hosting gives you full data control, at the cost of owning the controls). Vapi 3.6: fine for general commercial use, more friction in regulated deployments where call audio contains PII by definition — see PII handling in analytics for why "we'll redact it later" is not a plan.

Observability & evals

Weak across the board, which is the category's real gap. Vapi 4.1 has the best built-in call review of the four. Managed clouds 4.0 inherit their cloud's tracing. LiveKit 3.8, Pipecat 3.7 — both emit good traces and neither gives you a voice-specific eval harness. Everyone is building this themselves right now, and roughly 42% of production voice issues are voice-specific — invisible to transcript-only evaluation. Budget for it regardless of platform.

Ecosystem & integration

LiveKit 4.3 — the WebRTC ecosystem is deep and the SDK coverage is broad. Vapi 3.8, Pipecat 4.0 (Python means every AI library is already there). Managed clouds 3.9, excellent inside their boundary and indifferent outside it.

TCO & cost control

A mainstream cascaded stack runs roughly $0.08–$0.15 per conversation-minute plus about $0.013/min per telephony leg. The open frameworks let you pay that cost directly: Pipecat 4.2, LiveKit 4.0. Vapi adds platform margin on top of the same underlying vendors, which is entirely fair and entirely real: 2.9. Managed clouds 3.4 — bundled, predictable, rarely cheapest. The thing that dominates all of this at scale isn't the platform: it's that on a 30-minute call the LLM share of cost climbs to roughly 47% because history is re-sent every turn.

4. Radar scorecard

Comparator scores (same axis order) — Pipecat: 4.5 / 3.9 / 4.3 / 4.7 / 3.9 / 3.8 / 4.0 / 3.7 / 4.0 / 4.2. Vapi: 4.0 / 4.4 / 3.9 / 3.5 / 4.2 / 4.0 / 3.6 / 4.1 / 3.8 / 2.9. Managed cloud: 4.2 / 3.4 / 4.0 / 2.8 / 4.6 / 4.6 / 4.7 / 4.0 / 3.9 / 3.4.

5. The radar

radar-beta
  title Voice Agent Platform Radar
  axis Lat["Latency & Turns"], Tel["Telephony"], Tool["Tool Calling"], Flex["Model Flex"], Scale["Scale"], HA["HA / DR"], Gov["Governance"], Obs["Observability"], Eco["Ecosystem"], Cost["Cost Control"]
  curve LiveKit["LiveKit"]{ Lat: 4.6, Tel: 4.6, Tool: 4.2, Flex: 4.5, Scale: 4.4, HA: 4.2, Gov: 3.9, Obs: 3.8, Eco: 4.3, Cost: 4.0 }
  curve Pipecat["Pipecat"]{ Lat: 4.5, Tel: 3.9, Tool: 4.3, Flex: 4.7, Scale: 3.9, HA: 3.8, Gov: 4.0, Obs: 3.7, Eco: 4.0, Cost: 4.2 }
  curve Vapi["Vapi"]{ Lat: 4.0, Tel: 4.4, Tool: 3.9, Flex: 3.5, Scale: 4.2, HA: 4.0, Gov: 3.6, Obs: 4.1, Eco: 3.8, Cost: 2.9 }
  curve Managed["Managed cloud"]{ Lat: 4.2, Tel: 3.4, Tool: 4.0, Flex: 2.8, Scale: 4.6, HA: 4.6, Gov: 4.7, Obs: 4.0, Eco: 3.9, Cost: 3.4 }
  max 5
  min 0
  ticks 5
  graticule polygon
          

Interactive version — click a platform in the legend to toggle it on or off:

Read the shapes rather than the totals. LiveKit and Pipecat are nearly the same polygon rotated slightly: LiveKit bulges toward telephony and scale, Pipecat toward model flexibility and cost. Vapi's dent is entirely on the cost axis, and that dent is the product — you're buying time. The managed clouds are the odd shape: excellent everywhere the compliance team looks, weakest exactly where the engineers look.

6. Workload-specific analysis

7. Reference architecture

The pattern I'd build for the target use case: carrier into SIP, the agent as a participant in a room, the LLM behind a gateway so the middle of the pipeline stays swappable, tools exposed over MCP where the boundary is genuinely external, and every call recorded into an eval loop from day one rather than after the first bad quarter.

flowchart LR
  PSTN[PSTN / Carrier] --> SIP[SIP trunk]
  Web[Web / Mobile SDK] --> RTC[WebRTC]
  SIP --> Room["Media room
(SFU)"]
  RTC --> Room
  Room --> Agent["Agent worker
VAD + turn detector"]
  Agent --> STT[STT]
  STT --> GW["LLM gateway
routing + fallback"]
  GW --> LLM[LLM]
  LLM --> TTS[TTS]
  TTS --> Room
  LLM -.tool calls.-> MCP["MCP / internal APIs"]
  MCP -.-> CRM[(CRM / booking / KB)]
  Room --> Rec["Recording + transcript"]
  Rec --> Eval["Eval harness
golden + replay"]
  Rec --> Store[(Audit store)]
  PII[PII redaction] -.governs.-> Store
          

The configuration detail that matters most, and the one I'd put in the first commit — endpointing patience varying by what you just asked:

# Turn-detection patience by workflow step, not one global threshold.
# A pause after "what's your account number?" is almost certainly mid-utterance.
turn_detection:
  default:
    completion_threshold: 0.55     # semantic turn model probability
    max_silence_ms: 700
  steps:
    collect_account_number:
      completion_threshold: 0.80   # demand strong evidence the turn ended
      max_silence_ms: 2000         # people chunk digits
    collect_address:
      completion_threshold: 0.75
      max_silence_ms: 1800
    confirm_yes_no:
      completion_threshold: 0.45   # a short pause here IS the answer
      max_silence_ms: 400

barge_in:
  backchannel_phrases: ["mhm", "uh huh", "yeah", "right", "okay"]
  min_speech_ms: 220               # a click or cough must not stop playback
  truncate_context_on_interrupt: true   # model history = what was HEARD

8. POC plan (4 weeks)

• Week 1 — instrument before you build. Stand up the eval harness first: 20 golden conversations drawn from real call recordings, scored on task completion, P95 dead air, and clipped words. If you can't measure turn-taking on day one you will not measure it at all.
• Week 2 — one flow, two platforms. Build the single highest-volume call flow on LiveKit and on Vapi. Same STT, same LLM, same voice. You're comparing the pipeline, not the vendors underneath it.
• Week 3 — break it on purpose. Real PSTN, not WebRTC. Accented speech, background noise, someone reading a 16-digit number, an interruption mid-sentence, and a tool call you've deliberately slowed to 2 seconds. This is the week the answer usually becomes obvious.
• Week 4 — decide on numbers. Re-score this table with your measured latency, your missed-barge-in rate, and your actual cost per minute at your median call length. Write the ADR. Commit to Adopt / Narrow-Use / Defer.

9. Final recommendation

Recommendation: Trial LiveKit for voice-core products; Vapi for everything else.

Rationale:
  - LiveKit's native SIP removes a hop from the latency budget that matters
    most on the channel that matters most (the phone).
  - Pipecat is the better pick if you already run media infrastructure or
    need maximum model pluggability; the gap to LiveKit is small and the
    choice is mostly about what your team already knows.
  - Vapi's per-minute markup is cheaper than an engineer until it isn't.
    Do that arithmetic at your projected volume, not today's.
  - The managed clouds win only when data residency is a binding constraint.
    If it is, that's a real reason and the flexibility cost is worth paying.

Non-negotiables regardless of platform:
  - Build the eval harness before the agent. Voice failures are inaudible
    in transcripts and invisible in latency dashboards.
  - Keep the LLM behind a gateway. The middle of the pipeline WILL change.
  - Model cost at your p90 call length, not your median.

Reassess: when speech-to-speech tool calling matures. That's the change that
  would redraw this whole radar, and it's closer than it was a year ago.

Scores are my own architect-level judgment for a specific use case, calibrated to mid-2026 releases — not a vendor ranking. The axes differ from my analytics assessments, so scores are not comparable across pages. Re-score against your own POC numbers before you commit.