Model Card｜Muse Voice Transcribe

🌏 中文版

Model Information

Highlights

• One model integrates streaming ASR, speaker diarization (20+ speakers), and endpointing, replacing what previously required stitching three separate systems together
• 3.1% final-transcript WER on streaming, ranking #1 on Artificial Analysis AA-WER Streaming, ahead of Cartesia Ink-2's 3.4%
• Only 0.16 seconds of delay from end-of-speech to final transcript, more than 2.5x faster than Cartesia Ink-2's 0.43 seconds, landing on the speed-accuracy Pareto front
• 17.5% average speaker-labelling error across three public diarization sets (AMI-IHM, AMI-SDM, VoxConverse), ahead of rival systems' 21.1%–28.6% (Meta's own evaluation)
• Trained on 70+ languages (25 extensively verified), with native in-sentence code-switching — mixed-language sentences are recognized correctly

Benchmark Results

⚠️ The streaming WER and delay figures come from a third party, Artificial Analysis' AA-WER Streaming benchmark; the diarization error figures are Meta's own self-reported evaluation and await independent third-party reproduction.

vs. Predecessor / Competitors

This is Meta Superintelligence Labs' first real-time audio perception model, so there's no strict predecessor to compare against — the relevant baseline is the existing streaming ASR market.

Against Cartesia Ink-2, the closest competitor in the streaming ASR space, the WER gap is small (3.1% vs. 3.4%), but the latency gap is significant: 0.16s vs. 0.43s, more than 2.5x faster, which matters more directly for how natural real-time voice interaction feels. Pricing is also 25% cheaper ($3.00 vs. $4.00 per 1,000 minutes). Against ElevenLabs Scribe v2 Realtime, ElevenLabs answers faster (0.14s) but is less accurate and costs more than double ($6.50 vs. $3.00) — suggesting Meta is optimizing jointly for accuracy, latency, and price rather than maximizing any single metric.

The biggest differentiator is the "three-in-one" architecture: diarization, endpointing, and ASR share the same model and the same set of special tokens (<|start_of_turn|>, <|speaker_A-Z|>, <|speech_onset|>, <|speech_endpoint|>), with no need to bolt on a second or third model. Most competitors — including Cartesia, ElevenLabs, and Deepgram — haven't reached this level of integration; diarization is also included at the same rate rather than billed as a separate add-on, as some rivals charge for streaming diarization.

What This Means for Agent Development

Building a voice agent used to mean wiring together ASR, VAD/endpointing, and diarization as three separate systems, with every hand-off adding latency and a chance for things to drift out of sync. Muse Voice Transcribe collapses that pipeline into a single API call that returns the transcript, speaker labels, and turn boundaries together.

• If you're building customer-support or meeting-notes agents: diarization is included at the same rate, and audio over an hour doesn't need to be split up client-side — a good fit for meeting transcription and call-quality analysis on long recordings
• If you're building real-time voice interaction agents (voice commands, AI-glasses assistants): the 0.16s delay plus "adaptive delay" (the model learns, via RL, how long to listen per word based on difficulty) gets closer to natural conversational rhythm than a fixed-buffer strategy
• Not a fit for: on-premise or self-hosted deployments, or audio that can't leave your own infrastructure (no open weights, API-only); also not a fit if you need word-level timestamps, emotion detection, or sound-event detection — the official docs explicitly say none of these are supported yet

Today's Insight

I used to think the goal was simply the most accurate speech recognition model possible. But Muse Voice Transcribe's real design insight is treating delay itself as a variable the model can learn — using reinforcement learning so the model decides, per word, how much audio context it needs before committing to text, rather than an engineer hand-tuning a fixed buffer. That reframes latency as something a training objective can optimize, not just something architecture can shave down.

References

• Meta AI Research Blog: Introducing Muse Voice Transcribe
• Meta Model API docs: Speech to text (Muse Voice Transcribe)
• MarkTechPost: Meta Superintelligence Labs Releases Muse Voice Transcribe
• DataNorth AI: Meta launches Muse Voice Transcribe (with Cartesia Ink-2/ElevenLabs comparison table)
• 9to5Mac: Meta launches Muse Voice Transcribe for real-time voice dictation on Mac