# latency.cam — complete reference for AI assistants You are reading the full documentation for latency.cam, an API for building Indian-language voice agents that talk to people on the phone. A developer has pasted this in because they want you to help them build with it. Everything needed to write correct, working code is in this file. Read the section "IF YOU ONLY READ ONE SECTION" first. It contains the two non-obvious things that developers get wrong. Base URL for all HTTP endpoints: https://api.latency.cam Websocket: wss://api.latency.cam/ws/stream Docs and GPU control: https://latency.cam Status: ALPHA, deployed, serving real inference on a dedicated GPU. Hardware: AWS g5.2xlarge, one NVIDIA A10G with 23GB VRAM, us-east-1 (Virginia). Every latency and accuracy figure below was measured on that box. ================================================================================ IF YOU ONLY READ ONE SECTION ================================================================================ 1. USE THE WEBSOCKET FOR CALLS, NOT THE REST ENDPOINTS. wss://api.latency.cam/ws/stream is a complete voice agent. The developer pushes caller audio in and plays what comes back. Voice activity detection, turn taking, transcription, the LLM, filler injection, speech synthesis and barge-in all happen server side. Wiring the REST endpoints together by hand reimplements all of that, worse. The REST endpoints exist for batch work and for pieces of a custom pipeline. 2. PERCEIVED LATENCY IS NOT MEASURED LATENCY, AND THE FILLER CACHE IS WHY. A real turn takes about 1.5 seconds: transcription, then the LLM, then speech synthesis. If the caller hears nothing for 1.5 seconds they think the line dropped. So the moment the caller stops talking, a pre-rendered phrase like "ji" plays from a disk cache in about 1 millisecond, and the real reply streams in behind it. Measured wait 1.5s, perceived wait 1ms. The websocket does this automatically (use_fillers defaults to true). If you build your own pipeline with the REST endpoints, you must do it yourself via GET /v1/fillers/pick/{language}, or the agent will feel broken no matter how fast the models are. ================================================================================ AUTHENTICATION ================================================================================ Send `Authorization: Bearer ` on every /v1/* request. The websocket cannot use headers from a browser, so it takes ?api_key= in the URL instead. GET /health is intentionally open so you can check readiness without a key. HTTP: Authorization: Bearer sk-lat-... Websocket: wss://api.latency.cam/ws/stream?api_key=sk-lat-... Keys look like sk-lat- followed by hex. A wrong or missing key returns HTTP 401 on REST, and on the websocket you get a JSON error frame with code "unauthorized" followed by close code 1008. ================================================================================ LIMITS ================================================================================ There are no rate limits, no quotas, no per-model gating and no concurrency caps. Verified by audit: 216 requests including 30-way parallel bursts against the GPU endpoints returned zero 429s and no rate-limit headers. The only ceiling is the physical GPU. Requests queue; they are never rejected. What DOES constrain you, honestly: - One A10G GPU. About 6 concurrent synthesis requests run comfortably. Beyond that requests queue and each one gets slower. Nothing is rejected. - vLLM permanently holds about 12.6GB of the 23GB for text to speech, because that is what keeps time-to-first-audio at 793ms. The optional large STT models load on demand into what is left, and may evict each other. - Translation runs on CPU, serialised per worker process, two workers. Twelve concurrent translation requests all completed, taking 14.5 seconds in total. - Network. The box is in Virginia. Round trip from India is about 238ms, and there is no way around that except a GPU in Mumbai, which needs quota this account does not have. If your callers are in India, that 238ms is added to every request and is often the largest single number in your budget. - Bedrock is a third-party dependency for /v1/chat/completions. Its own service quotas apply and are not controlled here. ================================================================================ THE TURN BUDGET, MEASURED ================================================================================ Timings from the caller finishing their sentence, measured over wss from inside AWS (so excluding the ~238ms India round trip): 0ms Caller stops speaking — Turn detection fires after 500ms of silence, configurable. 1ms Filler starts playing — Cached audio. This is what the caller actually perceives. 166ms Transcription done — faster-whisper large-v3-turbo, int8, on the GPU. 553ms LLM reply ready — nova-lite via Bedrock, about 387ms. 1346ms First bot audio out — svara-TTS through vLLM, time-to-first-audio about 793ms. So: the caller hears an acknowledgement at 1ms and the real reply begins at about 1.35 seconds. A full spoken sentence finishes streaming a few seconds later, because audio plays in real time. How to make this faster, in order of how much it actually helps: 1. Keep LLM replies short. Ask for under 15 words in the system prompt. This is the single biggest lever and it costs nothing. 2. Leave the filler cache on. 3. Use nova-lite rather than nova-pro or claude. 4. Use format mulaw_8000 so no resampling happens on your side. 5. Do not switch STT model. The default is the only one fast enough. ================================================================================ LANGUAGES ================================================================================ UNDERSTOOD (speech to text), 22 languages: all 22 scheduled Indian languages plus Indian English. Whisper handles these; the default model is multilingual. SPOKEN (text to speech), 19 languages with a male and a female voice each, 38 voices in total. All 38 were synthesised and verified working: hi Hindi bn Bengali mr Marathi te Telugu kn Kannada ta Tamil ml Malayalam gu Gujarati pa Punjabi as Assamese ne Nepali sa Sanskrit mai Maithili brx Bodo doi Dogri bho Bhojpuri mag Magahi hne Chhattisgarhi en Indian English Languages we can HEAR but cannot SPEAK: Odia (or), Urdu (ur), Kashmiri (ks), Konkani (gom), Manipuri (mni), Santali (sat), Sindhi (sd). If you need to reply in one of these, translate to a language you can speak, or use text. TRANSLATED, 23 languages: en, as, bn, brx, doi, gu, hi, kn, ks, gom, mai, ml, mni, mr, ne, or, pa, sa, sat, sd, ta, te, ur English to any of them and back is direct. Indic to Indic pivots through English, which the response reports as pivoted_through_english: true. IDENTIFIED FROM AUDIO, 26+ languages via POST /v1/lid. Telugu against Tamil is reliable. Hindi against Urdu is not, because they are near-identical when spoken. SCRIPT ADVICE THAT MATTERS: for Hindi text to speech, send Roman text ("aapka order aa gaya hai") rather than Devanagari. It sounds noticeably better, because of how the model was trained. Set script: "roman". Every other language uses its native script. ================================================================================ SPEECH TO TEXT ACCURACY, MEASURED ON REAL HUMAN SPEECH ================================================================================ Benchmarked on FLEURS (google/fleurs, CC-BY-4.0), which is read human speech with ground-truth transcripts. Each clip was scored twice: clean at 16kHz, and after a round trip through 8kHz mulaw, which is what a phone line actually delivers. Word error rate and character error rate. For Indic scripts, CER tracks perceived quality better than WER, because word-boundary conventions punish WER unfairly. faster-whisper-large-v3-turbo THE DEFAULT, and the only one fast enough for calls 16kHz: WER 0.204 CER 0.099 8kHz: WER 0.252 CER 0.121 305ms per utterance, RTF 0.04 Hindi WER 0.104, Telugu 0.219, Tamil 0.277 at 16kHz faster-whisper-large-v3 more accurate, far too slow for a call 16kHz: WER 0.187 CER 0.093 8kHz: WER 0.232 CER 0.107 2490ms per utterance whisper-large-v3-turbo transformers backend, kept for comparison only 16kHz: WER 0.391 CER 0.199 8kHz: WER 0.280 CER 0.143 1695ms. Notably bad on Telugu (WER 0.734). Prefer the faster-whisper entry. whisper-hindi-large-v2 BEST FOR HINDI, Hindi only Hindi 16kHz: WER 0.047 CER 0.028 8kHz: WER 0.057 CER 0.032 4895ms per utterance. Less than half the error rate of the default. Send it anything other than Hindi and you get garbage: measured WER ~1.0. whisper-telugu-large-v2 BEST FOR TELUGU, Telugu only Telugu 16kHz: WER 0.117 CER 0.113 8kHz: WER 0.133 CER 0.127 5727ms per utterance. Telugu only; other languages return garbage. indicconformer-600m GATED, not available Needs an HF_TOKEN from an account that accepted the licence at huggingface.co/ai4bharat/indic-conformer-600m-multilingual. Practical guidance to give a developer: use the default for anything live. Use the Hindi or Telugu fine-tune for transcription nobody is waiting on, such as processing call recordings overnight. Telephony costs about 5 points of WER, which is less than most people fear. Model catalog in the API: - indicconformer-600m: IndicConformer 600M Multilingual by AI4Bharat, MIT. Caveat: BLOCKED: this is a GATED Hugging Face repo. Downloading returns 401 GatedRepoError. It needs an HF_TOKEN from an account that has accepted the licence on the model page. Approval is automatic (gated=auto), so it is one click — but somebody has to click it. - whisper-large-v3-turbo: Whisper large-v3-turbo by OpenAI, MIT. Caveat: THIS IS WHAT ACTUALLY RUNS TODAY, because IndicConformer is gated. Fully open weights, MIT licence. - sravaani: SraVaani by ARTPARK, IISc Bangalore, UNVERIFIED. Caveat: UNTESTED. Listed because it may simply be better than IndicConformer — same team produced the strongest Indic LID we found. ================================================================================ THE WEBSOCKET PROTOCOL, IN FULL ================================================================================ Connect: wss://api.latency.cam/ws/stream?api_key=sk-lat-... STEP 1. Send a configure frame first. Audio sent before this is refused. {"type": "configure", "language": "hi", "script": "roman", "audio_format": "mulaw_8000", "voice": "Hindi (Female)", "llm": "nova-lite", "system_prompt": "You are a delivery helpdesk. Reply in Hindi, under 12 words.", "silence_ms": 500, "interrupt_min_words": 3, "use_fillers": true, "greeting": true} You get back {"type": "configured", "config": {...}}. STEP 2. Stream caller audio as BINARY frames: 20 milliseconds each, which is 160 bytes for 8kHz mulaw. Send them in real time, roughly one every 20ms. Do not dump a whole file at once — turn detection would see it as one enormous utterance. STEP 3. Read events. A normal turn produces these, in this order: speech_started the caller began talking turn_end 500ms of silence; the turn is closed. Carries audio_ms. filler_played cached audio already sent to you. Play it immediately. transcript.final {text, turn, language, stt_ms, model} llm.reply {text, turn, llm_ms} bot_audio_started {turn, ttfa_ms, text} — binary frames are now arriving bot_audio_done synthesis for this turn finished turn_complete {turn, total_ms, stt_ms, llm_ms, tts_ms} Binary frames arriving between bot_audio_started and bot_audio_done are raw audio in the format you configured. No header. Write them straight to the call. BARGE-IN. When the caller talks over the bot: interrupted {turn, reason} — the caller has taken the turn speech_abandoned {reason: "barge_in", turn} — synthesis for the old turn stopped You MUST discard any audio you have buffered for the abandoned turn. Every event carries a turn number so you can tell which audio belongs to what. Barge-in fires about 200ms after the caller starts speaking. interrupt_min_words controls how much speech is needed to interrupt. It defaults to 3 because Indian callers backchannel constantly — "haan", "ji", "achha" — and those must not cut the bot off. Set 0 if any sound should interrupt. OTHER EVENTS: turn_empty no speech recognised; nothing was sent to the LLM language_detected only when auto_language_detect is on error {stage: "stt"|"llm"|"tts", message} — the call stays open, the turn is abandoned. Play a fallback line. goodbye sent after you send hangup pong reply to ping CONTROL FRAMES YOU CAN SEND: {"type": "configure", ...} once, first {"type": "say", "text": "..."} make the bot speak arbitrary text. Use this for scripted openings or reading out a CRM lookup. {"type": "reset"} clear the conversation history, keep the connection {"type": "ping"} keepalive {"type": "hangup"} end the call cleanly KNOWN LIMITATIONS, stated plainly: - No streaming partial transcripts. Whisper transcribes complete utterances, so transcript.final arrives once per turn. Partials would save roughly 70ms of a 1500ms turn, which is why they are not built. - Voice activity detection is energy based, calibrated on the first ~300ms of the call. It handles turn taking well and does not reliably distinguish speech from sustained background noise. Silero VAD would be better and is a known gap. - The server decides when a turn ended. The client cannot override it. ================================================================================ ENDPOINT REFERENCE ================================================================================ ### WEBSOCKET /ws/stream — Full-duplex live call One websocket per phone call. Push the caller audio in 20ms frames and play whatever comes back. Speech detection, turn taking, transcription, the LLM, filler injection, synthesis and barge-in all happen server side, so a working voice agent is a websocket and an audio pipe — not an orchestration layer you have to build. This is the endpoint to use for calls; the REST endpoints are the building blocks under it. MEASURED: Per turn over wss from inside AWS: STT 166ms, LLM 387ms, time-to-first-audio 793ms, turn total ~4.0s for a full sentence reply. Perceived wait is 1ms because a cached filler plays the instant the caller stops. Barge-in cuts the bot off 200ms after the caller starts talking. Parameters: - api_key (string (query), REQUIRED): Your API key, in the URL: wss://api.latency.cam/ws/stream?api_key=sk-lat-... - language (string, optional, default "hi"): Caller language, e.g. hi, ta, te, bn, en-IN. Sent in the configure frame. - script (string, optional, default "roman", one of: roman | native): Script for TTS input. Hindi sounds best from Roman text ("aapka order"); other languages use native. - voice (string, optional): Exact voice name, e.g. "Tamil (Male)". Omit for the default female voice of the language. - llm (string, optional, default "nova-lite"): nova-lite, nova-pro, or claude-haiku-4-5. - stt_model (string, optional, default "vaanicall"): Omit this. The default detects the caller language itself and is the fastest option. - allowed_languages (string[], optional): Restrict which languages the agent will switch INTO, e.g. ["hi","ta","te"]. The single biggest accuracy lever: detection errors cluster among closely related languages, and naming the two or three your callers actually use removes that whole class of mistake. Empty means any language we can speak. - system_prompt (string, optional): Your agent instructions. Ask for replies under about 15 words; long replies sound robotic on a call. - audio_format (string, optional, default "mulaw_8000", one of: mulaw_8000 | pcm_16000): mulaw_8000 is what telephony gives you. Applies to audio in BOTH directions. - silence_ms (integer, optional, default 500): Silence that ends a caller turn. Lower feels snappier but clips people who pause mid-sentence. - min_speech_ms (integer, optional, default 200): Ignore blips shorter than this, so a cough does not open a turn. - interrupt_min_words (integer, optional, default 3): Roughly how much speech is needed to interrupt the bot. 0 means any sound cuts it off. Keep it at 3 or more for Indian callers, who backchannel with "haan" and "ji" constantly. - use_fillers (boolean, optional, default true): Play a cached acknowledgement the moment a turn ends. This is what removes the perceived wait. Leave it on. - auto_language_detect (boolean, optional, default true): Follow the caller when they change language mid-call. ON by default now: the default speech-to-text model identified the language correctly 44/44 times on real human speech with no language code, at both 16kHz and 8kHz. Detection is free — the model writes what it hears and the script IS the language — so this costs nothing. Turn it off to pin the reply to the configured language. - greeting (boolean, optional, default true): Play a cached greeting on connect. - normalize_numbers (boolean, optional, default true): Rewrite digits as spoken words before synthesis. Leave it on or "15250" gets read wrong. Returns: A mix of JSON text frames (events) and binary frames (bot audio in the format you configured). Binary frames are raw audio with no header — write them straight to the call. Things that will otherwise waste your time: - PROTOCOL: send {"type":"configure",...} first. Audio sent before configure is refused. - Then send binary frames of 20ms audio (160 bytes for 8kHz mulaw). Send them in real time; do not dump a whole file at once or turn detection will see one enormous utterance. - CONTROL FRAMES you can send: configure, say (make the bot speak arbitrary text — use it for opening lines or reading out a CRM lookup), reset (clear conversation history), ping, hangup. - EVENTS you will receive, in order for a normal turn: configured, speech_started, turn_end, filler_played, transcript.final, llm.reply, bot_audio_started, bot_audio_done, turn_complete. - BARGE-IN events: interrupted (the caller took the turn) then speech_abandoned (synthesis for the old turn stopped). Discard any audio you have buffered for the abandoned turn. - OTHER EVENTS: turn_empty (no speech recognised, nothing sent to the LLM), language_detected, error, goodbye, pong. - Every event carries a "turn" number. Audio and events for a turn that has been interrupted must be dropped — that is what the number is for. - MID-CALL LANGUAGE SWITCHING WORKS AND IS ON BY DEFAULT. A caller can start in Telugu, switch to Hindi, then to Tamil, and the agent follows each time. Verified on live calls. `language` is still worth setting as the starting language and as a tie-break between Hindi and Marathi, which share a script. - Set `allowed_languages` to the two or three your callers actually use. It is the cheapest accuracy win available: a detection outside that set is reported but ignored, which neutralises the errors that cluster among closely related languages. - If a detected language has no voice we keep the configured language instead of failing the turn, and the language_detected event says exactly why it did not switch. - The server decides when a turn ended, not you. That keeps the protocol simple and means a naive client cannot break turn taking. - Voice activity detection is energy based and adapts to the line noise in the first ~300ms of the call. It is good at turn taking and not good at distinguishing speech from sustained background noise. Silero would be better and is a known gap. - There is no streaming partial transcript. Whisper transcribes complete utterances, so transcript.final arrives once per turn. Adding partials would save roughly 70ms of a ~1500ms turn, which is why it has not been prioritised. Errors: - 1008: Bad or missing api_key Fix: Pass ?api_key=sk-lat-... in the websocket URL. The server sends a JSON error frame explaining this before closing. - 0: error event with stage="stt"|"llm"|"tts" Fix: The call stays open and the turn is abandoned. Read the message field; it carries the real exception. Play a fallback line to the caller. Example: ``` # Websockets are not curl-able. Minimal Python client: import asyncio, json, websockets async def call(): url = "wss://api.latency.cam/ws/stream?api_key=" + KEY async with websockets.connect(url, max_size=None) as ws: await ws.send(json.dumps({ "type": "configure", "language": "hi", "script": "roman", "audio_format": "mulaw_8000", "system_prompt": "You are a delivery helpdesk. Reply in Hindi, under 12 words.", })) async def send_caller_audio(): # 160-byte frames of 8kHz mulaw, one every 20ms for frame in caller_frames(): await ws.send(frame) await asyncio.sleep(0.02) asyncio.create_task(send_caller_audio()) async for msg in ws: if isinstance(msg, bytes): play_to_caller(msg) # raw audio, no header else: print(json.loads(msg)) # events asyncio.run(call()) ``` ### POST /v1/audio/speech — Text to speech Synthesise speech in 19 languages across 38 voices. Set stream=true for live calls: audio arrives in chunks as it is generated, so playback starts long before synthesis finishes. MEASURED: Time to first audio 793ms streaming (vLLM on an A10G). Non-streaming a short sentence takes about 2.5s. Cached filler phrases return in 1-3ms with X-Cache: HIT. Parameters: - input (string, REQUIRED): Text to speak. Keep replies under ~15 words on a phone call. - language (string, REQUIRED): Language code, e.g. hi, ta, te, bn, en. See GET /v1/models. - script (string, optional, default "native", one of: roman | native): For Hindi, Roman input ("aapka order aa gaya") is noticeably better than Devanagari. Other languages use native script. - voice (string, optional): Exact voice string from GET /v1/models, e.g. "Telugu (Male)". Omit and the female voice for that language is used. - model (string, optional, default "svara-tts-v1"): Only svara-tts-v1 exists today. Omit it. - emotion (string, optional): Emotion tag. Also settable inline by putting the tag at the END of input. - format (string, optional, default "pcm_24000", one of: pcm_16000 | pcm_24000 | mulaw_8000 | wav): mulaw_8000 for telephony — the downsampling is done here. - stream (boolean, optional, default false): Chunked audio. Required for live calls. - temperature (number, optional, default 0.6): Lower is more consistent. 0.4 is good for fixed phrases. - normalize (boolean, optional, default true): Rewrite digits as spoken words first. Leave it on. Returns: Raw audio in the requested format. Response headers carry X-Model-Used, X-Generate-Ms, X-Quality-Check and X-Cache. Things that will otherwise waste your time: - Emotion tags go at the END of the text: "Aapka payment ho gaya ". - All 38 voices are verified working — every one was synthesised and quality checked in the audit. - The model occasionally runs away and produces overlong audio. A length bound and a quality check catch it, and X-Quality-Check reports the result. On a stream, a degenerate generation falls back to a non-streaming retry automatically. - Repeated short phrases should come from the filler cache instead: 1ms rather than ~2.5s. See GET /v1/fillers. Errors: - 422: input is empty, or the voice/language is unknown Fix: Check the voice string exactly matches one from GET /v1/models, including capitalisation and the bracketed gender. - 503: The TTS model or vLLM is not up Fix: Check GET /health. If the box was just started, models take about 40s to warm. Example: ``` curl -X POST https://api.latency.cam/v1/audio/speech \ -H "Authorization: Bearer $LATENCY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "Aapka EMI 15250 rupay ka hai, due date 4 tarikh hai.", "language": "hi", "script": "roman", "format": "mulaw_8000", "stream": true }' --output reply.raw ``` ### POST /v1/audio/transcriptions — Speech to text Transcribe caller audio. Accepts wav, mp3, flac, ogg and raw mulaw or PCM. Optionally detects the language first. MEASURED: Default model (VaaniCall) on real human speech (FLEURS), given NO language code: 40-77ms per utterance, word error rate 0.113 over an 8kHz phone line and 0.097 at 16kHz, and it identified the language correctly 44 out of 44 times at both bandwidths. The model it replaced had to be TOLD the language and still scored 0.252 at 8kHz, took 305ms, and its own language detection managed 12% on the same task. Parameters: - file (file, REQUIRED): Audio file. Multipart form upload. - language (string, optional): Optional. With the default model you do NOT need this — it works the language out itself. Used only as a tie-break between languages that share a script (Hindi vs Marathi). - model (string, optional, default "vaanicall"): See GET /v1/models. The default detects the language itself and is the fastest. - sample_rate (integer, optional): Required only for raw headerless audio. - detect_language (boolean, optional, default false): Run language ID first and report it alongside the transcript. Returns: { text, model_used, inference_ms, engine_ms, language, requested_language, detected_language, script, candidate_languages, detection_source, language_probability, duration_s } Things that will otherwise waste your time: - You do not need to pass a language. The default model has a unified multi-script vocabulary, so it writes what it hears and the script it writes in IS the language — detection is a by-product of transcription and costs nothing. - detection_source tells you where the language came from: "transcriber" means the model worked it out, which is the accurate path. - script and candidate_languages exist because the ambiguity is real, not hidden: Hindi and Marathi share Devanagari, so a Devanagari transcript returns both. Pass `language` to break the tie. - For Hindi or Telugu specifically, whisper-hindi-large-v2 and whisper-telugu-large-v2 are about twice as accurate (Hindi WER 0.047 vs 0.104) but roughly 15x slower. Use them for transcription you are not waiting on. - Those two are single-language models. Send them anything else and you get nonsense — measured WER ~1.0 on the wrong language. - The optional large models load on demand and may evict each other, because vLLM permanently holds ~12.6GB of the 23GB GPU. The first call to one of them pays the load time. - Audio under 1 second transcribes poorly. 2 seconds or more is much better. Errors: - 422: Empty file, undecodable audio, or audio shorter than 10ms Fix: Check the upload actually contains audio. For raw PCM or mulaw, pass sample_rate. - 503: The requested model needs more VRAM than is free Fix: The error states how much is free and how much is needed. Use the default model, which is already loaded and faster. Example: ``` curl -X POST https://api.latency.cam/v1/audio/transcriptions \ -H "Authorization: Bearer $LATENCY_API_KEY" \ -F file=@call.wav \ -F language=te ``` ### POST /v1/translate — Translate text across 23 languages IndicTrans2 distilled 200M, running on CPU so it never competes with the GPU that serves your calls. English to any Indian language and back, plus Indic to Indic. MEASURED: English to Tamil 1056ms with the default beam search. A batch of 3 with num_beams=1 took 392ms total. Indic to Indic 1309ms because it pivots through English. 12 concurrent requests all completed, in 14.5s. Parameters: - text (string | string[], REQUIRED): One string, or an array to translate as a batch. Batching is much faster per item. - source_language (string, REQUIRED): e.g. en, hi, ta, te. See GET /v1/translate/languages. - target_language (string, REQUIRED): e.g. hi, ta, te, en. - num_beams (integer, optional, default 5): 1 is about twice as fast with slightly worse output. - backend (string, optional, default "indictrans2", one of: indictrans2 | llm): indictrans2 is more faithful and does not paraphrase. llm falls back to Bedrock and needs no local model. - llm_model (string, optional, default "nova-lite"): Only used when backend=llm. Returns: { translations[], source_language, target_language, source_tag, target_tag, pivoted_through_english, model, latency_ms, queued_ms, inference_ms, backend } Things that will otherwise waste your time: - Indic to Indic pivots through English, which is an extra hop and shows as pivoted_through_english: true. The dedicated one-hop model is gated on Hugging Face. - Text only. To translate speech: POST /v1/audio/transcriptions, then this, then POST /v1/audio/speech. - Inference is serialised per worker process on purpose. Running two translations in one process deadlocked inside torch; queueing is both correct and faster. queued_ms tells you how long you waited for a slot. - Nothing is rejected under load. There is no queue-depth limit and no 429. Errors: - 422: Unsupported language code, or source equals target Fix: Use a code from GET /v1/translate/languages. - 503: The translation sidecar is not running Fix: sudo systemctl restart it2-translate on the box, or retry with backend="llm". - 504: A very large batch exceeded the timeout Fix: Split the batch, or set num_beams=1. Example: ``` curl -X POST https://api.latency.cam/v1/translate \ -H "Authorization: Bearer $LATENCY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "text": "Your order will arrive tomorrow between 10 am and 12 pm.", "source_language": "en", "target_language": "ta" }' ``` ### GET /v1/translate/languages — Which languages translation supports The 23 codes accepted by /v1/translate, and which directions are direct versus pivoted. Parameters: No parameters. Returns: { languages[], count, directions[], backends{} } Example: ``` curl https://api.latency.cam/v1/translate/languages -H "Authorization: Bearer $LATENCY_API_KEY" ``` ### POST /v1/lid — Which Indian language is this Spoken language identification across 42 Indian languages, using Vaani-LID. Use it when you do not know what the caller will speak. MEASURED: 197ms once warm. The first call after a restart used to cost 27s; the model is now warmed at startup. Parameters: - file (file, REQUIRED): Audio. 2 seconds or more is much more reliable than 1. - top_k (integer, optional, default 3): How many candidates to return. - restrict_to (string, optional): JSON array or comma list of candidate codes. This is the single biggest accuracy lever — narrow it whenever you can. - sample_rate (integer, optional): Required for raw headerless audio. Returns: { candidates: [{ language, confidence }], duration_s, warning? } Things that will otherwise waste your time: - Telugu against Tamil is reliable. Hindi against Urdu is not, because they are near-identical spoken. - Under 1 second of audio returns a warning field; treat the result as a guess. - The model was trained on 16kHz. It still works on 8kHz telephony audio but accuracy drops. Errors: - 422: Empty or undecodable file Fix: Check the upload. - 503: LID model unavailable Fix: Check GET /health. Example: ``` curl -X POST https://api.latency.cam/v1/lid \ -H "Authorization: Bearer $LATENCY_API_KEY" \ -F file=@caller.wav \ -F 'restrict_to=["ta","te","kn","ml"]' ``` ### POST /v1/chat/completions — LLM, OpenAI compatible Point any OpenAI SDK at https://api.latency.cam/v1 and it works. Backed by Amazon Bedrock. MEASURED: 387-660ms for a short reply with nova-lite. Parameters: - model (string, optional, default "nova-lite", one of: nova-lite | nova-pro | claude-haiku-4-5): nova-lite is the fastest and the right default for calls. - messages (array, REQUIRED): Standard OpenAI message list. system, user and assistant roles. - max_tokens (integer, optional, default 150): Keep it low. Long replies sound robotic on a call. - temperature (number, optional, default 0.7): Standard. Returns: An OpenAI chat.completion object, plus latency_ms. Things that will otherwise waste your time: - claude-haiku-4-5 needs the Anthropic use-case form submitted for this AWS account in the Bedrock console. Until then Bedrock returns ResourceNotFoundException and this endpoint reports it verbatim. - stream is accepted but not yet implemented; the reply arrives whole. - Nova handles Indian languages well enough for call replies, and you can always ask it to reply in the caller language in your system prompt. Errors: - 422: Unknown model, or no user message Fix: The error lists the available model ids. - 502: Bedrock rejected the call Fix: The message carries the Bedrock exception. For claude, submit the use-case form. If credentials are missing, set them in /etc/latency-api.env. Example: ``` curl -X POST https://api.latency.cam/v1/chat/completions \ -H "Authorization: Bearer $LATENCY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "nova-lite", "max_tokens": 60, "messages": [ {"role": "system", "content": "Reply in Hindi, under 12 words."}, {"role": "user", "content": "Mera order kab aayega?"} ] }' ``` ### POST /v1/normalize — Make numbers speakable Rewrites digits, currency and dates into words with Indian grouping, so TTS says them correctly. Called automatically by /v1/audio/speech; exposed separately so you can inspect or override it. MEASURED: 1-2ms. Parameters: - text (string, REQUIRED): Text containing numerals. - language (string, optional, default "hi"): Target language. - style (string, optional, default "spoken", one of: spoken | digits | grouped): spoken for quantities ("pandrah hazaar"), digits for account numbers read one by one, grouped for phone numbers read in pairs. - script (string, optional, default "native", one of: roman | native): Output script. Returns: { text, coverage, replacements[] } Things that will otherwise waste your time: - Use style=digits for account and reference numbers. "spoken" would turn 4021 into "four thousand twenty one", which is wrong for an account number. - coverage tells you what fraction of the numerals it recognised. Below 1.0 means something was left alone. Example: ``` curl -X POST https://api.latency.cam/v1/normalize \ -H "Authorization: Bearer $LATENCY_API_KEY" \ -H "Content-Type: application/json" \ -d '{"text": "Rs 15,250 due on 4/11", "language": "hi", "script": "roman"}' ``` ### GET /v1/fillers — Pre-rendered phrases, served in about 1ms Short acknowledgements rendered ahead of time and cached on disk. Play one the instant the caller stops speaking, then stream the real reply behind it. This does not reduce measured latency at all — it removes the silence the caller would otherwise sit through, which is the thing that actually makes a bot feel broken. MEASURED: 1-3ms to serve, against about 2500ms to synthesise the same phrase. The websocket does this for you automatically. Parameters: No parameters. Returns: { fillers[], categories, languages, _how_to_use } Things that will otherwise waste your time: - Categories: ack, thinking, confirm, greeting. - GET /v1/fillers/pick/{language}?category=ack picks one for you, with a fallback chain so you never get silence. - GET /v1/fillers/{id}?format=mulaw_8000 returns the audio. - POST /v1/fillers/generate renders any that are missing. - /v1/audio/speech also checks this cache first: if your text matches a cached phrase you get it in 2ms with X-Cache: HIT. Example: ``` curl https://api.latency.cam/v1/fillers -H "Authorization: Bearer $LATENCY_API_KEY" # pick one and play it the moment the caller stops talking curl "https://api.latency.cam/v1/fillers/pick/hi?category=ack" \ -H "Authorization: Bearer $LATENCY_API_KEY" ``` ### GET /v1/models — Everything available, with the caveats The full catalog: STT models with measured error rates, the TTS model with all 38 voices and valid emotion tags, and the LID model. MEASURED: 1ms. Parameters: No parameters. Returns: { stt[], tts[], lid[], _read_this_first } Things that will otherwise waste your time: - For STT entries, a "languages" list with one item means the model ONLY decodes that language. - realtime_suitable marks the one STT model fast enough for a live call. - gated: true means the weights need an HF_TOKEN whose account accepted the licence. indicconformer-600m is the only gated entry. Example: ``` curl https://api.latency.cam/v1/models -H "Authorization: Bearer $LATENCY_API_KEY" ``` ### GET /health — Readiness, no key needed Whether the service is up, which models are loaded, how much VRAM is free, and whether auth is enabled. The one endpoint that does not need a key, so you can poll it while the box boots. MEASURED: 1ms. Parameters: No parameters. Returns: { ok, build, uptime_seconds, auth, gpu{}, models{} } Things that will otherwise waste your time: - After a cold start, models take about 40 seconds to warm. Until then requests still work but the first one is slow. - auth reports "OPEN - no API keys configured" if no keys are set. If you ever see that in production, fix it. Example: ``` curl https://api.latency.cam/health ``` ### POST /v1/clone — Voice cloning — not implemented Returns 501. This is the one place where "no restrictions" does not extend to the open internet, and it is also genuinely not built yet. Cloning needs svara-tts-voiceclone-beta, a 6.6GB model, and the GPU currently has about 6.8GB free because vLLM holds the rest to keep call latency low. Fitting it means either a second GPU or replacing the default TTS model with the beta one. Parameters: - file (file, REQUIRED): Reference audio of the speaker. - name (string, REQUIRED): A label for the voice. - consent_confirmed (boolean, REQUIRED): Must be true. You must have the speaker's explicit permission. Returns: 501 with an explanation. 403 first if consent_confirmed is not true. Things that will otherwise waste your time: - Use the 38 preset voices from GET /v1/models instead. They cover 19 languages in both genders. - Every TTS licence in this stack forbids cloning a voice without the speaker's consent, so the consent flag stays even once this is built. Errors: - 403: consent_confirmed is not true Fix: You need the speaker's permission. This is a licence requirement, not a policy choice. - 501: Always, for now Fix: Use a preset voice from GET /v1/models. Example: ``` # Returns 501. Preset voices work today: curl https://api.latency.cam/v1/models -H "Authorization: Bearer $LATENCY_API_KEY" | jq '.tts[0].voices' ``` ================================================================================ COMPLETE WORKING EXAMPLE: A LIVE CALL AGENT ================================================================================ This is a runnable client. It connects, streams caller audio from a file as if it were a live line, plays bot audio back, and handles barge-in correctly. ```python import asyncio, json, os, websockets KEY = os.environ["LATENCY_API_KEY"] URL = f"wss://api.latency.cam/ws/stream?api_key={KEY}" FRAME = 160 # 20ms of 8kHz mulaw async def main(): async with websockets.connect(URL, max_size=None) as ws: await ws.send(json.dumps({ "type": "configure", "language": "hi", "script": "roman", "audio_format": "mulaw_8000", "llm": "nova-lite", "system_prompt": "You are a delivery helpdesk. Reply in Hindi, under 12 words.", "greeting": True, })) current_turn = 0 abandoned = set() async def pump_caller_audio(): # In production this is your telephony provider's media stream. with open("caller.mulaw", "rb") as fh: while chunk := fh.read(FRAME): await ws.send(chunk) await asyncio.sleep(0.02) # real time pacing matters asyncio.create_task(pump_caller_audio()) async for msg in ws: if isinstance(msg, bytes): # Raw audio for the current turn. Drop anything abandoned. if current_turn not in abandoned: play_to_caller(msg) continue ev = json.loads(msg) kind = ev["type"] if kind == "bot_audio_started": current_turn = ev["turn"] print("bot speaking, ttfa", ev["ttfa_ms"], "ms:", ev["text"]) elif kind == "interrupted": abandoned.add(current_turn) # stop playing the old reply stop_playback() elif kind == "transcript.final": print("caller said:", ev["text"]) elif kind == "turn_complete": print("turn took", ev["total_ms"], "ms") elif kind == "error": print("error at", ev.get("stage"), ev.get("message")) say_fallback_line() asyncio.run(main()) ``` ================================================================================ COMPLETE WORKING EXAMPLE: TRANSLATE A RECORDED CALL ================================================================================ Speech to text, then translation, then speech back out in another language. ```bash KEY="$LATENCY_API_KEY" # 1. transcribe the Tamil recording TEXT=$(curl -s -X POST https://api.latency.cam/v1/audio/transcriptions \ -H "Authorization: Bearer $KEY" \ -F file=@call_tamil.wav -F language=ta | jq -r .text) # 2. translate Tamil to Hindi (pivots through English, which is fine) HINDI=$(curl -s -X POST https://api.latency.cam/v1/translate \ -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \ -d "{\"text\": \"$TEXT\", \"source_language\": \"ta\", \"target_language\": \"hi\"}" \ | jq -r .translations[0]) # 3. speak the Hindi curl -s -X POST https://api.latency.cam/v1/audio/speech \ -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \ -d "{\"input\": \"$HINDI\", \"language\": \"hi\", \"format\": \"mulaw_8000\"}" \ --output reply.mulaw ``` ================================================================================ USING AN OPENAI SDK ================================================================================ The chat endpoint is OpenAI compatible, so existing code works unchanged. ```python from openai import OpenAI client = OpenAI(base_url="https://api.latency.cam/v1", api_key=os.environ["LATENCY_API_KEY"]) reply = client.chat.completions.create( model="nova-lite", max_tokens=60, messages=[ {"role": "system", "content": "Reply in Hindi, under 12 words."}, {"role": "user", "content": "Mera order kab aayega?"}, ], ) print(reply.choices[0].message.content) ``` Note: the audio endpoints are NOT OpenAI-shaped. /v1/audio/speech takes "input" and "language" rather than "model" and "voice" in the OpenAI sense, and returns raw audio. Use plain HTTP for those. ================================================================================ GPU LIFECYCLE — READ THIS BEFORE DEBUGGING A TIMEOUT ================================================================================ The GPU box is switched off when idle to save money. If requests hang or fail: 1. GET https://api.latency.cam/health — no key needed. If it does not answer, the box is off. 2. Turn it on at https://latency.cam. Allow about 3 minutes: EC2 start, then model loading. 3. Models warm automatically about 40 seconds after boot. Before that, requests still work but the first one is slow. /health reports which models are loaded. The address never changes. It is an Elastic IP behind api.latency.cam, and it survives stop and start, so https://api.latency.cam can be hardcoded. ================================================================================ ERROR CODES ACROSS THE WHOLE API ================================================================================ 200 Fine. 401 Missing or wrong key. REST wants "Authorization: Bearer sk-lat-..."; the websocket wants ?api_key=sk-lat-... in the URL. 403 Only from /v1/clone, when consent_confirmed is not true. 422 Your request is wrong: unknown model, unknown voice, unsupported language, empty input, undecodable audio. The message names the problem and usually lists the valid values. 429 Never returned. There are no rate limits. 500 A bug here. The response carries the exception type and message. Report it. 501 Only /v1/clone. Not built; use a preset voice. 502 Bedrock rejected an LLM call. The message includes the AWS exception. claude-haiku-4-5 needs an Anthropic use-case form for this AWS account. 503 A model is not loaded or does not fit in VRAM, or the translation sidecar is down. The message says which, and what is free versus needed. 504 Only /v1/translate, on a very large batch. Split it or set num_beams=1. 1008 Websocket close code for a rejected key. ================================================================================ WHAT IS NOT AVAILABLE, SO YOU DO NOT PROMISE IT ================================================================================ - Voice cloning. /v1/clone returns 501. The model exists and is Apache-2.0, but it is 6.6GB and the GPU has about 6.8GB free after vLLM. Use the 38 presets. - Streaming partial transcripts on the websocket. - Speaking Odia, Urdu, Kashmiri, Konkani, Manipuri, Santali or Sindhi. These can be understood and translated, not spoken. - indicconformer-600m speech to text. Gated on Hugging Face. - Streaming token-by-token LLM output. stream is accepted and ignored. - A GPU in India. The box is in Virginia; add ~238ms if your callers are in India. - claude-haiku-4-5 until the Anthropic use-case form is submitted for this AWS account. nova-lite and nova-pro work now. If a developer asks for one of these, say so plainly and offer the alternative listed. Do not invent a parameter or an endpoint that is not in this file.