Endpoints
Audio and LLM endpoints live on api.latency.cam. Docs, the model catalog and GPU control live on latency.cam and stay up even when the GPU is off.
wss://api.latency.cam/ws/stream is a complete voice agent — turn taking, transcription, the LLM, filler injection, synthesis and barge-in all run server side. The REST endpoints are the building blocks under it, useful for batch work and custom pipelines. Full protocol →Authentication
Authorization: Bearer <YOUR_API_KEY> # every /v1/* endpoint wss://api.latency.cam/ws/stream?api_key=... # the websocket
One header unlocks every endpoint. No rate limits, no quotas, no per-model gating, no concurrency caps — verified across 216 requests including 30-way parallel bursts, with zero 429s. The websocket takes the key in the URL because browsers cannot set headers on a WebSocket. GET /health needs no key at all.
Every error code, and what to do
| Code | Meaning | Fix |
|---|---|---|
401 | Missing or bad key | Check the Authorization header, or ?api_key= on the websocket |
403 | Only /v1/clone, when consent_confirmed is not true | You need the speaker's permission. Licence requirement, not policy. |
422 | Bad request: unknown model or voice, unsupported language, empty or undecodable input | The message names the problem and usually lists the valid values |
429 | Never returned | There are no rate limits |
500 | A bug here | The response carries the exception type and message. Report it. |
501 | Only /v1/clone. Not built. | Use one of the 38 preset voices |
502 | Bedrock rejected an LLM call | Message includes the AWS exception. claude-haiku-4-5 needs the Anthropic use-case form; nova-lite works now. |
503 | A model is not loaded, does not fit in VRAM, or the translation sidecar is down | The message says which, and how much VRAM is free versus needed. Check /health. If the box just started, models warm in about 40s. |
504 | Only /v1/translate, on a very large batch | Split the batch, or set num_beams: 1 |
1008 | Websocket close code for a rejected key | A JSON error frame with the reason arrives before the close |
WEBSOCKET /ws/stream alpha
One websocket per phone call. Push the caller audio in 20ms frames and play whatever comes back. Speech detection, turn taking, transcription, the LLM, filler injection, synthesis and barge-in all happen server side, so a working voice agent is a websocket and an audio pipe — not an orchestration layer you have to build. This is the endpoint to use for calls; the REST endpoints are the building blocks under it.
Parameters
| Name | Type | Default | Description |
|---|---|---|---|
api_key req | string (query) | — | Your API key, in the URL: wss://api.latency.cam/ws/stream?api_key=sk-lat-... |
language | string | hi | Caller language, e.g. hi, ta, te, bn, en-IN. Sent in the configure frame. |
script | string | roman | Script for TTS input. Hindi sounds best from Roman text ("aapka order"); other languages use native. One of: roman native |
voice | string | — | Exact voice name, e.g. "Tamil (Male)". Omit for the default female voice of the language. |
llm | string | nova-lite | nova-lite, nova-pro, or claude-haiku-4-5. |
stt_model | string | vaanicall | Omit this. The default detects the caller language itself and is the fastest option. |
allowed_languages | string[] | — | Restrict which languages the agent will switch INTO, e.g. ["hi","ta","te"]. The single biggest accuracy lever: detection errors cluster among closely related languages, and naming the two or three your callers actually use removes that whole class of mistake. Empty means any language we can speak. |
system_prompt | string | — | Your agent instructions. Ask for replies under about 15 words; long replies sound robotic on a call. |
audio_format | string | mulaw_8000 | mulaw_8000 is what telephony gives you. Applies to audio in BOTH directions. One of: mulaw_8000 pcm_16000 |
silence_ms | integer | 500 | Silence that ends a caller turn. Lower feels snappier but clips people who pause mid-sentence. |
min_speech_ms | integer | 200 | Ignore blips shorter than this, so a cough does not open a turn. |
interrupt_min_words | integer | 3 | Roughly how much speech is needed to interrupt the bot. 0 means any sound cuts it off. Keep it at 3 or more for Indian callers, who backchannel with "haan" and "ji" constantly. |
use_fillers | boolean | true | Play a cached acknowledgement the moment a turn ends. This is what removes the perceived wait. Leave it on. |
auto_language_detect | boolean | true | Follow the caller when they change language mid-call. ON by default now: the default speech-to-text model identified the language correctly 44/44 times on real human speech with no language code, at both 16kHz and 8kHz. Detection is free — the model writes what it hears and the script IS the language — so this costs nothing. Turn it off to pin the reply to the configured language. |
greeting | boolean | true | Play a cached greeting on connect. |
normalize_numbers | boolean | true | Rewrite digits as spoken words before synthesis. Leave it on or "15250" gets read wrong. |
Returns
A mix of JSON text frames (events) and binary frames (bot audio in the format you configured). Binary frames are raw audio with no header — write them straight to the call.
- PROTOCOL: send {"type":"configure",...} first. Audio sent before configure is refused.
- Then send binary frames of 20ms audio (160 bytes for 8kHz mulaw). Send them in real time; do not dump a whole file at once or turn detection will see one enormous utterance.
- CONTROL FRAMES you can send: configure, say (make the bot speak arbitrary text — use it for opening lines or reading out a CRM lookup), reset (clear conversation history), ping, hangup.
- EVENTS you will receive, in order for a normal turn: configured, speech_started, turn_end, filler_played, transcript.final, llm.reply, bot_audio_started, bot_audio_done, turn_complete.
- BARGE-IN events: interrupted (the caller took the turn) then speech_abandoned (synthesis for the old turn stopped). Discard any audio you have buffered for the abandoned turn.
- OTHER EVENTS: turn_empty (no speech recognised, nothing sent to the LLM), language_detected, error, goodbye, pong.
- Every event carries a "turn" number. Audio and events for a turn that has been interrupted must be dropped — that is what the number is for.
- MID-CALL LANGUAGE SWITCHING WORKS AND IS ON BY DEFAULT. A caller can start in Telugu, switch to Hindi, then to Tamil, and the agent follows each time. Verified on live calls. `language` is still worth setting as the starting language and as a tie-break between Hindi and Marathi, which share a script.
- Set `allowed_languages` to the two or three your callers actually use. It is the cheapest accuracy win available: a detection outside that set is reported but ignored, which neutralises the errors that cluster among closely related languages.
- If a detected language has no voice we keep the configured language instead of failing the turn, and the language_detected event says exactly why it did not switch.
- The server decides when a turn ended, not you. That keeps the protocol simple and means a naive client cannot break turn taking.
- Voice activity detection is energy based and adapts to the line noise in the first ~300ms of the call. It is good at turn taking and not good at distinguishing speech from sustained background noise. Silero would be better and is a known gap.
- There is no streaming partial transcript. Whisper transcribes complete utterances, so transcript.final arrives once per turn. Adding partials would save roughly 70ms of a ~1500ms turn, which is why it has not been prioritised.
Errors
| Code | When | Fix |
|---|---|---|
1008 | Bad or missing api_key | Pass ?api_key=sk-lat-... in the websocket URL. The server sends a JSON error frame explaining this before closing. |
0 | error event with stage="stt"|"llm"|"tts" | The call stays open and the turn is abandoned. Read the message field; it carries the real exception. Play a fallback line to the caller. |
Example
# Websockets are not curl-able. Minimal Python client:
import asyncio, json, websockets
async def call():
url = "wss://api.latency.cam/ws/stream?api_key=" + KEY
async with websockets.connect(url, max_size=None) as ws:
await ws.send(json.dumps({
"type": "configure",
"language": "hi", "script": "roman",
"audio_format": "mulaw_8000",
"system_prompt": "You are a delivery helpdesk. Reply in Hindi, under 12 words.",
}))
async def send_caller_audio():
# 160-byte frames of 8kHz mulaw, one every 20ms
for frame in caller_frames():
await ws.send(frame)
await asyncio.sleep(0.02)
asyncio.create_task(send_caller_audio())
async for msg in ws:
if isinstance(msg, bytes):
play_to_caller(msg) # raw audio, no header
else:
print(json.loads(msg)) # events
asyncio.run(call())POST /v1/audio/speech alpha
Synthesise speech in 19 languages across 38 voices. Set stream=true for live calls: audio arrives in chunks as it is generated, so playback starts long before synthesis finishes.
Parameters
| Name | Type | Default | Description |
|---|---|---|---|
input req | string | — | Text to speak. Keep replies under ~15 words on a phone call. |
language req | string | — | Language code, e.g. hi, ta, te, bn, en. See GET /v1/models. |
script | string | native | For Hindi, Roman input ("aapka order aa gaya") is noticeably better than Devanagari. Other languages use native script. One of: roman native |
voice | string | — | Exact voice string from GET /v1/models, e.g. "Telugu (Male)". Omit and the female voice for that language is used. |
model | string | svara-tts-v1 | Only svara-tts-v1 exists today. Omit it. |
emotion | string | — | Emotion tag. Also settable inline by putting the tag at the END of input. |
format | string | pcm_24000 | mulaw_8000 for telephony — the downsampling is done here. One of: pcm_16000 pcm_24000 mulaw_8000 wav |
stream | boolean | false | Chunked audio. Required for live calls. |
temperature | number | 0.6 | Lower is more consistent. 0.4 is good for fixed phrases. |
normalize | boolean | true | Rewrite digits as spoken words first. Leave it on. |
Returns
Raw audio in the requested format. Response headers carry X-Model-Used, X-Generate-Ms, X-Quality-Check and X-Cache.
- Emotion tags go at the END of the text: "Aapka payment ho gaya <happy>".
- All 38 voices are verified working — every one was synthesised and quality checked in the audit.
- The model occasionally runs away and produces overlong audio. A length bound and a quality check catch it, and X-Quality-Check reports the result. On a stream, a degenerate generation falls back to a non-streaming retry automatically.
- Repeated short phrases should come from the filler cache instead: 1ms rather than ~2.5s. See GET /v1/fillers.
Errors
| Code | When | Fix |
|---|---|---|
422 | input is empty, or the voice/language is unknown | Check the voice string exactly matches one from GET /v1/models, including capitalisation and the bracketed gender. |
503 | The TTS model or vLLM is not up | Check GET /health. If the box was just started, models take about 40s to warm. |
Example
curl -X POST https://api.latency.cam/v1/audio/speech \
-H "Authorization: Bearer $LATENCY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"input": "Aapka EMI 15250 rupay ka hai, due date 4 tarikh hai.",
"language": "hi",
"script": "roman",
"format": "mulaw_8000",
"stream": true
}' --output reply.rawPOST /v1/audio/transcriptions alpha
Transcribe caller audio. Accepts wav, mp3, flac, ogg and raw mulaw or PCM. Optionally detects the language first.
Parameters
| Name | Type | Default | Description |
|---|---|---|---|
file req | file | — | Audio file. Multipart form upload. |
language | string | — | Optional. With the default model you do NOT need this — it works the language out itself. Used only as a tie-break between languages that share a script (Hindi vs Marathi). |
model | string | vaanicall | See GET /v1/models. The default detects the language itself and is the fastest. |
sample_rate | integer | — | Required only for raw headerless audio. |
detect_language | boolean | false | Run language ID first and report it alongside the transcript. |
Returns
{ text, model_used, inference_ms, engine_ms, language, requested_language, detected_language, script, candidate_languages, detection_source, language_probability, duration_s }
- You do not need to pass a language. The default model has a unified multi-script vocabulary, so it writes what it hears and the script it writes in IS the language — detection is a by-product of transcription and costs nothing.
- detection_source tells you where the language came from: "transcriber" means the model worked it out, which is the accurate path.
- script and candidate_languages exist because the ambiguity is real, not hidden: Hindi and Marathi share Devanagari, so a Devanagari transcript returns both. Pass `language` to break the tie.
- For Hindi or Telugu specifically, whisper-hindi-large-v2 and whisper-telugu-large-v2 are about twice as accurate (Hindi WER 0.047 vs 0.104) but roughly 15x slower. Use them for transcription you are not waiting on.
- Those two are single-language models. Send them anything else and you get nonsense — measured WER ~1.0 on the wrong language.
- The optional large models load on demand and may evict each other, because vLLM permanently holds ~12.6GB of the 23GB GPU. The first call to one of them pays the load time.
- Audio under 1 second transcribes poorly. 2 seconds or more is much better.
Errors
| Code | When | Fix |
|---|---|---|
422 | Empty file, undecodable audio, or audio shorter than 10ms | Check the upload actually contains audio. For raw PCM or mulaw, pass sample_rate. |
503 | The requested model needs more VRAM than is free | The error states how much is free and how much is needed. Use the default model, which is already loaded and faster. |
Example
curl -X POST https://api.latency.cam/v1/audio/transcriptions \ -H "Authorization: Bearer $LATENCY_API_KEY" \ -F file=@call.wav \ -F language=te
POST /v1/translate alpha
IndicTrans2 distilled 200M, running on CPU so it never competes with the GPU that serves your calls. English to any Indian language and back, plus Indic to Indic.
Parameters
| Name | Type | Default | Description |
|---|---|---|---|
text req | string | string[] | — | One string, or an array to translate as a batch. Batching is much faster per item. |
source_language req | string | — | e.g. en, hi, ta, te. See GET /v1/translate/languages. |
target_language req | string | — | e.g. hi, ta, te, en. |
num_beams | integer | 5 | 1 is about twice as fast with slightly worse output. |
backend | string | indictrans2 | indictrans2 is more faithful and does not paraphrase. llm falls back to Bedrock and needs no local model. One of: indictrans2 llm |
llm_model | string | nova-lite | Only used when backend=llm. |
Returns
{ translations[], source_language, target_language, source_tag, target_tag, pivoted_through_english, model, latency_ms, queued_ms, inference_ms, backend }
- Indic to Indic pivots through English, which is an extra hop and shows as pivoted_through_english: true. The dedicated one-hop model is gated on Hugging Face.
- Text only. To translate speech: POST /v1/audio/transcriptions, then this, then POST /v1/audio/speech.
- Inference is serialised per worker process on purpose. Running two translations in one process deadlocked inside torch; queueing is both correct and faster. queued_ms tells you how long you waited for a slot.
- Nothing is rejected under load. There is no queue-depth limit and no 429.
Errors
| Code | When | Fix |
|---|---|---|
422 | Unsupported language code, or source equals target | Use a code from GET /v1/translate/languages. |
503 | The translation sidecar is not running | sudo systemctl restart it2-translate on the box, or retry with backend="llm". |
504 | A very large batch exceeded the timeout | Split the batch, or set num_beams=1. |
Example
curl -X POST https://api.latency.cam/v1/translate \
-H "Authorization: Bearer $LATENCY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"text": "Your order will arrive tomorrow between 10 am and 12 pm.",
"source_language": "en",
"target_language": "ta"
}'GET /v1/translate/languages alpha
The 23 codes accepted by /v1/translate, and which directions are direct versus pivoted.
Returns
{ languages[], count, directions[], backends{} }
Example
curl https://api.latency.cam/v1/translate/languages -H "Authorization: Bearer $LATENCY_API_KEY"
POST /v1/lid alpha
Spoken language identification across 42 Indian languages, using Vaani-LID. Use it when you do not know what the caller will speak.
Parameters
| Name | Type | Default | Description |
|---|---|---|---|
file req | file | — | Audio. 2 seconds or more is much more reliable than 1. |
top_k | integer | 3 | How many candidates to return. |
restrict_to | string | — | JSON array or comma list of candidate codes. This is the single biggest accuracy lever — narrow it whenever you can. |
sample_rate | integer | — | Required for raw headerless audio. |
Returns
{ candidates: [{ language, confidence }], duration_s, warning? }
- Telugu against Tamil is reliable. Hindi against Urdu is not, because they are near-identical spoken.
- Under 1 second of audio returns a warning field; treat the result as a guess.
- The model was trained on 16kHz. It still works on 8kHz telephony audio but accuracy drops.
Errors
| Code | When | Fix |
|---|---|---|
422 | Empty or undecodable file | Check the upload. |
503 | LID model unavailable | Check GET /health. |
Example
curl -X POST https://api.latency.cam/v1/lid \ -H "Authorization: Bearer $LATENCY_API_KEY" \ -F file=@caller.wav \ -F 'restrict_to=["ta","te","kn","ml"]'
POST /v1/chat/completions alpha
Point any OpenAI SDK at https://api.latency.cam/v1 and it works. Backed by Amazon Bedrock.
Parameters
| Name | Type | Default | Description |
|---|---|---|---|
model | string | nova-lite | nova-lite is the fastest and the right default for calls. One of: nova-lite nova-pro claude-haiku-4-5 |
messages req | array | — | Standard OpenAI message list. system, user and assistant roles. |
max_tokens | integer | 150 | Keep it low. Long replies sound robotic on a call. |
temperature | number | 0.7 | Standard. |
Returns
An OpenAI chat.completion object, plus latency_ms.
- claude-haiku-4-5 needs the Anthropic use-case form submitted for this AWS account in the Bedrock console. Until then Bedrock returns ResourceNotFoundException and this endpoint reports it verbatim.
- stream is accepted but not yet implemented; the reply arrives whole.
- Nova handles Indian languages well enough for call replies, and you can always ask it to reply in the caller language in your system prompt.
Errors
| Code | When | Fix |
|---|---|---|
422 | Unknown model, or no user message | The error lists the available model ids. |
502 | Bedrock rejected the call | The message carries the Bedrock exception. For claude, submit the use-case form. If credentials are missing, set them in /etc/latency-api.env. |
Example
curl -X POST https://api.latency.cam/v1/chat/completions \
-H "Authorization: Bearer $LATENCY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "nova-lite",
"max_tokens": 60,
"messages": [
{"role": "system", "content": "Reply in Hindi, under 12 words."},
{"role": "user", "content": "Mera order kab aayega?"}
]
}'POST /v1/normalize alpha
Rewrites digits, currency and dates into words with Indian grouping, so TTS says them correctly. Called automatically by /v1/audio/speech; exposed separately so you can inspect or override it.
Parameters
| Name | Type | Default | Description |
|---|---|---|---|
text req | string | — | Text containing numerals. |
language | string | hi | Target language. |
style | string | spoken | spoken for quantities ("pandrah hazaar"), digits for account numbers read one by one, grouped for phone numbers read in pairs. One of: spoken digits grouped |
script | string | native | Output script. One of: roman native |
Returns
{ text, coverage, replacements[] }
- Use style=digits for account and reference numbers. "spoken" would turn 4021 into "four thousand twenty one", which is wrong for an account number.
- coverage tells you what fraction of the numerals it recognised. Below 1.0 means something was left alone.
Example
curl -X POST https://api.latency.cam/v1/normalize \
-H "Authorization: Bearer $LATENCY_API_KEY" \
-H "Content-Type: application/json" \
-d '{"text": "Rs 15,250 due on 4/11", "language": "hi", "script": "roman"}'GET /v1/fillers alpha
Short acknowledgements rendered ahead of time and cached on disk. Play one the instant the caller stops speaking, then stream the real reply behind it. This does not reduce measured latency at all — it removes the silence the caller would otherwise sit through, which is the thing that actually makes a bot feel broken.
Returns
{ fillers[], categories, languages, _how_to_use }
- Categories: ack, thinking, confirm, greeting.
- GET /v1/fillers/pick/{language}?category=ack picks one for you, with a fallback chain so you never get silence.
- GET /v1/fillers/{id}?format=mulaw_8000 returns the audio.
- POST /v1/fillers/generate renders any that are missing.
- /v1/audio/speech also checks this cache first: if your text matches a cached phrase you get it in 2ms with X-Cache: HIT.
Example
curl https://api.latency.cam/v1/fillers -H "Authorization: Bearer $LATENCY_API_KEY" # pick one and play it the moment the caller stops talking curl "https://api.latency.cam/v1/fillers/pick/hi?category=ack" \ -H "Authorization: Bearer $LATENCY_API_KEY"
GET /v1/models alpha
The full catalog: STT models with measured error rates, the TTS model with all 38 voices and valid emotion tags, and the LID model.
Returns
{ stt[], tts[], lid[], _read_this_first }
- For STT entries, a "languages" list with one item means the model ONLY decodes that language.
- realtime_suitable marks the one STT model fast enough for a live call.
- gated: true means the weights need an HF_TOKEN whose account accepted the licence. indicconformer-600m is the only gated entry.
Example
curl https://api.latency.cam/v1/models -H "Authorization: Bearer $LATENCY_API_KEY"
GET /health alpha
Whether the service is up, which models are loaded, how much VRAM is free, and whether auth is enabled. The one endpoint that does not need a key, so you can poll it while the box boots.
Returns
{ ok, build, uptime_seconds, auth, gpu{}, models{} }
- After a cold start, models take about 40 seconds to warm. Until then requests still work but the first one is slow.
- auth reports "OPEN - no API keys configured" if no keys are set. If you ever see that in production, fix it.
Example
curl https://api.latency.cam/health
POST /v1/clone planned
Returns 501. This is the one place where "no restrictions" does not extend to the open internet, and it is also genuinely not built yet. Cloning needs svara-tts-voiceclone-beta, a 6.6GB model, and the GPU currently has about 6.8GB free because vLLM holds the rest to keep call latency low. Fitting it means either a second GPU or replacing the default TTS model with the beta one.
Parameters
| Name | Type | Default | Description |
|---|---|---|---|
file req | file | — | Reference audio of the speaker. |
name req | string | — | A label for the voice. |
consent_confirmed req | boolean | — | Must be true. You must have the speaker's explicit permission. |
Returns
501 with an explanation. 403 first if consent_confirmed is not true.
- Use the 38 preset voices from GET /v1/models instead. They cover 19 languages in both genders.
- Every TTS licence in this stack forbids cloning a voice without the speaker's consent, so the consent flag stays even once this is built.
Errors
| Code | When | Fix |
|---|---|---|
403 | consent_confirmed is not true | You need the speaker's permission. This is a licence requirement, not a policy choice. |
501 | Always, for now | Use a preset voice from GET /v1/models. |
Example
# Returns 501. Preset voices work today: curl https://api.latency.cam/v1/models -H "Authorization: Bearer $LATENCY_API_KEY" | jq '.tts[0].voices'