Benchmarks

Measured on our own hardware — NVIDIA A10G (g5.2xlarge), us-east-1. Nothing here is estimated. Measured 2026-09-05T14:30:00Z.

Turn latency: 3,135 ms → 1,488 msA 2.1× improvement, almost entirely from moving text-to-speech onto vLLM. With a pre-rendered filler played first, the caller perceives about 50 ms.

The whole surface, verified by probing it

Every number below comes from an audit script that sends real requests, not from reading the code.

endpoints working        14
endpoints broken         0
endpoints not built      1   (/v1/clone, honest 501)
voices producing audio   38/38
speech-to-text models    5/5 usable (1 more is gated)
websocket /ws/stream     WORKS

No rate limits, demonstrated

216 requests, 0 HTTP 429sIncluding 20-way and 30-way parallel bursts and parallel hits on every GPU endpoint. No rate-limit headers of any kind were returned. Requests queue when the hardware is busy; they are never rejected.

Probe	Requests	Parallel	Status codes
normalize serial	40	—	`200`
normalize 20-way parallel	60	20	`200`
models serial	30	—	`200`
filler cache 30-way parallel	60	30	`200`
TTS 8-way parallel (GPU)	8	8	`200`
STT 6-way parallel (GPU)	6	6	`200`
chat 6-way parallel (Bedrock)	6	6	`200`
translate 6-way parallel (CPU)	6	6	`200`

Live call, measured over the websocket

wss://api.latency.cam, from inside AWS (excludes the ~238ms India round trip)

transcription         166 ms
LLM reply             387 ms
time to first audio   793 ms
full turn            3997 ms   whole streamed sentence
perceived wait          1 ms   cached filler
barge-in              200 ms   after the caller starts talking

turn_total is the whole streamed sentence. What the caller experiences is the filler at 1ms and the reply starting at ~1.35s.

Speech-to-text accuracy on real human speech

google/fleurs test split, CC-BY-4.0. Read human speech with ground-truth transcripts. Scored at 16kHz and again after a round trip through 8kHz mulaw, which is what a phone line delivers. CER matters more than WER for Indic scripts because word-break conventions punish WER unfairly.

Model	16kHz WER / CER	8kHz WER / CER	Latency
`faster-whisper-large-v3-turbo`	0.204 / 0.099	0.252 / 0.121	345 ms
`faster-whisper-large-v3`	0.187 / 0.093	0.232 / 0.107	2490 ms
`whisper-large-v3-turbo`	0.391 / 0.199	0.28 / 0.143	1695 ms
`whisper-hindi-large-v2`	0.717 / 0.552	0.759 / 0.64	4895 ms
`whisper-telugu-large-v2`	0.686 / 0.47	0.691 / 0.471	5727 ms

The two single-language fine-tunes are the most accurate option for their language by a wide margin — Hindi WER 0.047 against 0.104 for the default — and roughly 15× slower. Use them for transcription nobody is waiting on. They only decode their own language; anything else returns garbage.

The 8kHz telephony penalty: WER 0.204 → 0.252About 5 points of WER. Real, but far less than commonly feared. This was the largest open question in earlier drafts of the docs and is now measured.

Translation

English to Tamil             1056 ms
Indic to Indic               1309 ms   pivots through English
batch of 3, num_beams=1       392 ms   total, not each
12 concurrent requests       14.5 s    all succeeded

Indic to Indic pivots through English, which is the extra ~250ms.

Turn breakdown

speech to text           220 ms   faster-whisper int8
LLM (nova-lite)          485 ms   Bedrock
normalize                  1 ms   no model
TTS first audio          782 ms   svara via vLLM
                      ────────
TOTAL                   1488 ms

before this work        3135 ms

What changed, and by how much

Component	Before	After	Gain
TTS first audio (streaming)	2,473 ms	782 ms	3.16×
Speech to text	307 ms	220 ms	1.4×
Concurrent capacity	~1 call	~6 calls measured	6.02×
Repeated phrases (fillers)	~2,400 ms	1.2 ms	~1,200×

The vLLM time-to-first-audio result

782 ms median, with a range of 782–783 ms. A 1 ms spread across runs, which is unusually stable for autoregressive generation and makes the number safe to design around.

But real-time factor is still above 1.0vLLM 1.282 versus in-process 2.099 — a 1.64× throughput gain, but both above 1.0. even on vllm, svara cannot sustain continuous real-time speech on an a10g. Streaming hides this because playback starts before generation finishes, and it stops mattering once the caller is listening.

Concurrency

The result that decides cost per call. Note how the vLLM per-request median barely moves as load rises — that is continuous batching working.

Concurrent	vLLM median	vLLM wall	in-process median	in-process wall
1	3,639 ms	3.6 s	6,529 ms	6.5 s
3	4,385 ms	4.5 s	16,738 ms	18.5 s
6	4,242 ms	13.9 s	82,246 ms	83.7 s

vLLM reports 21.99× maximum internal concurrency with a 4.81 GB KV cache. We measured 6 in parallel with zero errors; the ceiling is higher than we pushed it.

The network tax nobody sees

GPU region	Round trip from India
us-east-1 (Virginia) — current	238 ms
ap-south-1 (Mumbai)	25 ms

213 ms per turn, thrown away, and invisible in every on-box benchmark above. It is a launch problem rather than a testing one, but it is the single largest remaining item and no amount of code fixes it.

Memory and cold start

VRAM total	23.69 GB
vLLM (weights + KV cache + graphs)	13.1 GB
Free for the API's other models	9.2 GB
vLLM cold boot	147 s
vLLM warm cache	32 s

Verified across a real instance stop/start: every service came back unattended. Allow about three minutes after switching the box on before the API answers.

Reliability, stated plainly

svara does not always stop talkingsvara often fails to emit end-of-speech and generates until capped, WITHOUT erroring. Mitigated by a text-aware token budget (1.6x expected duration), an output sanity check, and up to 3 retries on the non-streaming path. Worst case bounded from 17.07s to 11.01s.

3 of 15 pre-rendered fillers were initially silent or rambling and had to be regenerated. Silence does not raise, so cached audio must be audited.

This is the most important operational fact on this page. A model that fails by returning plausible-looking silence, rather than by raising, has to be checked on every call — which is what the sanity check and retry now do.