Hacker News

How We Made a Text-to-Speech Model Respond in Sub-50 ms

22 points by toebee ago | 5 comments

toebee |next [-]

time-to-first-audio (TTFA) is critical for realtime voice applications. open source implementations (e.g. vLLM-Omni, SGLang-Omni) are often too slow for production and can have issues with realtime playback if you push for lower latency. we wanted to fix that.

we optimized qwen3-tts, a popular OSS TTS model, to achieve 34 ms p95 TTFA at 10 requests per second on 1 x H100. we open source the implementation and benchmark, as well as a breakdown of how it was done.

github: https://github.com/nari-labs/nari-qwen3-tts

kamranjon |root |parent |next [-]

Hi there! I actually thought your Dia models were amazing and very natural sounding, I haven’t tried qwen 3 tts yet - has your focus shifted away from building your Dia models and shifted more towards hosting and infrastructure?

narrationbox |root |parent |next |previous [-]

Haven't read the full report yet, just a quick question. Are your numbers for cold start without pre fill or is it after warmed cache?

bityard |root |parent |previous [-]

How fast is it on consumer-level hardware?

dominotw |previous [-]

chatgpt responds super fast but says filler words like 'hmm..' 'let me think' and responds later with delay