Tencent Open-Sources Hy-SimulMT, a Speech Interpreter That Replies in About 3 Seconds

Tencent's Hunyuan team released and open-sourced Hy-SimulMT on Oct. 26, a simultaneous interpretation model that turns Chinese speech into English speech, and back again, while the speaker is still talking. Weights and code are live on Hugging Face, ModelScope and GitHub, and the model debuted free inside the Hy Translate app.

The pitch is voice, not just text. Hy-SimulMT streams the audio of the speaker, translates while listening and outputs translated speech cloned in the original voice and tone, so listeners hear the rendition in something close to the voice of the speaker. Tencent demonstrates multi-speaker scenes in which three different voices are each cloned separately, a setup that maps directly onto panels and meetings.

Latency is the headline metric. Tencent measures the model with StreamLAAL, a standard streaming-translation lag metric, and reports about 3 seconds on long-form speech, with the human comparison run of its own posting 3.3 seconds against 4.5 seconds for CATTI Level 1 interpreters and 4.7 seconds for Level 2. The model uses a speak-first, correct-later strategy that mimics how human interpreters self-correct mid-sentence. Tencent notes StreamLAAL is not the same as the wait for the first translated word.

Quality numbers come from company-run evaluations and should be read as such. On the long-form RealSI benchmark scored with GEMBA under a GPT6.1-Sol judge, Hy-SimulMT was the best of the open models compared and ahead of most commercial APIs; on short sets it scored 90.3 on FLEURS, 81.2 on ACL6060 and 84.7 on RealSI-Sentence, the highest of every listed system, while Gemini 3.5-LiveTranslate kept the top FLEURS score at 92.4. In a small company-organized comparison across 18 recordings of about six minutes each in 10 domains, six CATTI-certified interpreters took part, and the model matched the best human quality scores at lower latency.

The release is a 30B-A3B mixture-of-experts system under Apache 2.0, with FP8 and GGUF builds and an Interleave variant that also emits the source transcript. Tencent ships SimulInfer, a serving stack built on vLLM with a web interface and WebSocket API, plus SimulTrain for domain tuning. Support in llama.cpp has not yet been merged.

The competitive frame is an arms race in live interpreting: Alibaba's Qwen3-LiveTranslate-Flash cut simultaneous interpretation latency earlier this year, and commercial APIs from Gemini, ByteDance Seed and iFlytek all compete in the same benchmarks. Wang Weiwei, vice dean of the interpreting school at Guangdong University of Foreign Studies, said the speak-then-correct style pushes machine interpreting toward usable communication in low-risk settings.

What it means: open weights at 3-second latency put real-time interpreting inside local deployments where cloud APIs cannot go, from compliance-bound enterprises to offline venues. High-stakes diplomacy and courtrooms still belong to humans, but meetings, lectures and livestreams are now squarely in machine territory.