Speech-to-speech, fine-tuned to talk like a human
A hybrid architecture — persona-driven, full-duplex speech-to-speech with live function calling — running at ~200ms practical latency (160ms theoretical), near human conversation speed.
Built for real, live conversation
Every improvement targets what breaks voice agents in production — not benchmark leaderboards.
Function Calling
Invokes tools and APIs mid-conversation — spoken requests become structured actions in real time. Base Moshi-style models can't do this at all.
Knows When To Talk
Most speech-to-speech models wait out a silence timer to know when to respond. Ours listens and speaks at the same time, and turns naturally like a human would.
Interruption Handling
Detects user barge-in and yields immediately without losing context or repeating itself.
Emotion & Tone Capture
Picks up on emotion, tone, hesitation, laughter, and other non-verbal sound — not just the transcript.
Persona + function calling, in one model
A hybrid of NVIDIA's persona-prompted Moshi and Moshi RAG — persona-based prompting plus real function calling, which neither base model does alone.
The fastest turn-taking we’ve measured
A static benchmark snapshot — our internal harness against public 24h medians from the voicebenchmark.ai leaderboard. Fixed numbers, not a live feed.
Every measured turn landed between 577 and 847ms — the shaded core is mean ±σ (613–753ms).
Barge-in isn’t a failure — it’s the model detecting a caller talking over it and handing the floor back without losing context.
How we fine-tuned it
Three passes, each targeting a specific failure mode we saw in live calls.
Start from persona Moshi
NVIDIA's persona-prompted Moshi gives natural voice, emotion, and tone — but no function calling or dynamic scripting on its own.
Add Moshi RAG for tools
Layered in Moshi RAG so the model can call functions and follow dynamic scripts mid-conversation, something neither base model does alone.
Optimize the serving path
Streaming inference and pipeline tuning brought end-to-end response time down to ~200ms practical, 160ms theoretical.
Hear it. Then call it.
A walkthrough video and a live in-browser call are both on the way — you’ll be able to push on latency and interruptions yourself.
Start a Live Web Call
Talk to the model directly in your browser — no phone number needed.
Everything's on Hugging Face
Cloned it, ran it locally in under ten minutes — latency numbers matched the model card exactly.