The Best Voice AI Research Papers of 2026 (So Far): Four Worth Reading, and Where Each One Is Weak

The strongest voice AI papers of 2026 so far split into two camps. Synthesis is nearly saturated on standard metrics: Xiaomi's OmniVoice reports a 1.30 word error rate on LibriSpeech-PC, below the 1.87 of the ground-truth recordings in the same table. Conversation is not: in the HumDial study, the top challenge entry scored 76.6 out of 100.

This is a selection, not a ranking. The four papers below were picked because each asks a different question (how to synthesise, how to score, how models behave) and each publishes enough method detail to check. All numbers are the authors' own and none has been independently replicated as of 15 October 2026.

Which voice AI papers of 2026 are worth reading?

Four arXiv papers cover the main layers of the field: speech synthesis, two ways to benchmark full-duplex dialogue, and a behavioural study of what those models actually do. Two other releases are mentioned briefly at the end, with a caveat about how much of them was read.

What does OmniVoice claim, and does the evidence support it?

OmniVoice is a zero-shot text-to-speech model that, according to its authors at Xiaomi Corp., scales to over 600 languages. It is a 0.8B-parameter, non-autoregressive model that maps text directly to multi-codebook acoustic tokens, instead of the usual two-stage text-to-semantic-to-acoustic pipeline. Two training choices matter: a full-codebook random masking strategy, and initialisation from a pretrained LLM for intelligibility.

The training corpus is 581k hours of multilingual audio curated entirely from open-source data. The authors use a speech restoration model to clean degraded audio and rule-based filters to drop invalid transcripts, and they upsample low-resource languages to offset imbalance.

Results from the paper's Table 1 (LibriSpeech-PC test-clean, baselines run from official checkpoints, all self-reported):

On speed, the paper reports a real-time factor of 0.0319 at 16 inference steps and batch size 1 on an H20 GPU, against 0.0557 for ZipVoice under the same setup. On a 24-language benchmark, the authors say OmniVoice beats ElevenLabs Multilingual v2 and MiniMax-Speech on average speaker similarity and WER. Code and pretrained models are public at github.com/k2-fsa/OmniVoice.

The abstract says state-of-the-art across Chinese, English and multilingual benchmarks. The paper's own tables are more mixed. On the English Seed-TTS test set, Qwen3-TTS has a lower WER (1.54 vs 1.60) and a higher UTMOS (4.16 vs 3.91). On the Chinese set, CosyVoice3 edges OmniVoice on speaker similarity (0.778 vs 0.777). The authors also note that a high Cantonese WER comes from the Whisper ASR model, and that it falls to 2.273% with SenseVoice-Small.

The practical point: when a model's WER sits below the ground-truth recordings, WER no longer separates systems well. The more informative result here is the breadth of coverage, with the caveat that the authors' own limitations section says open-source data is uneven in quality and the model is not tuned for complex numeric sequences. The ethics statement says the model is intended for academic research only and flags misuse risk from high speaker similarity.

What does the HumDial study add for full-duplex evaluation?

HumDial supplies a dataset, a benchmark and a leaderboard for interruption handling. The authors (Chengyou Wang, Lei Xie and colleagues) release more than 100 hours of human-recorded, dual-channel speech in Chinese and English, built from DeepSeek-generated scripts performed by professional actors. The benchmark, HumDial-FDBench, extends Full-Duplex-Bench v1.5.

Scoring has three parts. Interruption handling counts only a "Respond" reply as correct. Rejection (ignoring backchannels, pauses, third-party speech and speech aimed at others) counts only "Resume". Latency is scored on a log scale. The final score is 0.4 interruption plus 0.4 rejection plus 0.2 latency. DeepSeek-V3 classifies each reply from time-aligned ASR transcripts.

Two patterns stand out. First, rejection is the hard part: only one entry scored above 70 there. Second, most submissions were cascaded or semi-cascaded pipelines, with a lightweight front end deciding turn-taking. The authors say end-to-end systems offer low latency and prosody awareness but suffer "auditory blindness" while generating speech.

What does M3-DuplexBench show about dialogue context and language?

M3-DuplexBench, from Ryo Fukuda and colleagues at NTT, tests full-duplex models in English and Japanese, in casual chat and task-oriented question answering, under three context settings. None gives no history, User gives only the user's earlier speech, and Full conditions on both sides, with system turns fixed to reference speech by teacher forcing. The aim is a fair comparison across models in multi-turn conversation.

Dialogue history changed timing a lot. In the task-oriented domain, Moshi's smooth turn-taking latency fell from 2.453 seconds (None) to 0.430 seconds (Full), and J-Moshi's from 0.933 to 0.326 seconds. The effect on content was mixed: in English chat, Moshi's response relevance dropped from 0.781 to 0.703 under Full context while its context consistency rose from 0.818 to 0.887.

The authors also report that Japanese models underperformed English models, and that the gap lay mainly in language understanding and generation, not timing. They conclude that user-only conditioning can introduce context mismatch, and that full-context conditioning is the more reliable protocol.

Do full-duplex models speak up when they should?

Mostly not, according to "Full-Duplex Speech Models Take the Floor When Asked, Not When Needed" by Linkai Peng and colleagues. Across five model families, being addressed and silence were far more reliable triggers for speech than false facts or hazards.

The design is a matched-context test: English monologues in which only the trigger sentence changes, across 10 trigger conditions and 40 topics, with five seeds per stimulus, or 2,000 trials per configuration (Moshi pools five seeds from each of its checkpoints). Seven configurations were tested with default decoding: Moshi, Moshika-RL, PersonaPlex, PPlex-RL, NVIDIA VoiceChat-11B, MiniCPM-o 4.5 and Raon-SpeechChat.

The headline numbers concern Moshi and PersonaPlex once they have the floor. They answer most direct questions, but the share of non-empty false-fact replies that challenge the claim is only .14 to .15, and the share of hazard replies that warn of danger is .04 to .07. The authors say it remains open whether the models fail to notice the problem or notice it and stay silent. Stimuli, outputs and evaluation code are released at github.com/vocaliodmiku/take-the-floor.

Do these papers conflict, or ask different questions?

They mostly ask different questions, and they are complementary. HumDial and M3-DuplexBench both build benchmarks, but they target different failure modes. HumDial scores whether a system interrupts, resumes or ignores side speech correctly. M3-DuplexBench scores how history shifts timing and content across languages. "Take the Floor" cites HumDial as a benchmark of user-initiated events and asks about the opposite direction: when should the model start talking unprompted.

One real tension: HumDial's top entries are cascaded pipelines with explicit turn-taking modules, while M3-DuplexBench could only test Moshi-based end-to-end models because its Full setting needs direct conditioning on parallel speech streams. Cascaded systems are left to future work, so the two benchmarks cannot yet be compared on the same systems.

What should a working researcher take from this?

Four practical points follow from the papers (this section is the site's analysis).

• Do not trust one benchmark. Scores depend on what is measured: Gemini-2.5 scored 79.8 on HumDial interruptions but 36.5 on rejection. Report several.
• Fix the context condition. M3-DuplexBench shows latency for the same model can move from 2.453 to 0.430 seconds depending on dialogue history. A latency number without its context setting is incomplete.
• Test content-driven intervention. If your application needs a model to correct a user or warn of danger, the matched-context protocol in "Take the Floor" is a ready template, and its code is public.
• Check sample sizes and judges. Several of these evaluations rely on LLM judges and ASR transcripts. For a related discussion of small-trial evaluation in another field, see Most Robot Papers Test on 10 to 30 Trials.

What other 2026 voice releases are worth watching?

Two releases are relevant context, though they are described here from their abstracts only. The Qwen Team's Qwen3.5-Omni technical report (arXiv 2604.15804, submitted 17 April 2026) says the model was trained on over 100 million hours of audio-visual content and that its Qwen3.5-Omni-plus variant achieves state-of-the-art results across 215 audio and audio-visual subtasks. That is a lab-reported claim from the abstract, not a result confirmed against the report's tables.

PersonaPlex (arXiv 2602.06053) adds voice and role control to full-duplex models through hybrid prompting and large-scale synthetic training data. Its authors claim it surpasses earlier duplex models on role adherence, speaker similarity, latency and naturalness. It is built on the Moshi architecture, per M3-DuplexBench, and it is one of the models that "Take the Floor" found rarely challenges false claims.

What's still unclear?

• No replication. Every score above comes from the papers' own authors. None of the four papers reports an independent reproduction.
• Judge and data effects. HumDial uses scripted, actor-performed speech and an LLM (DeepSeek-V3) to classify replies. Errors in ASR or the judge flow straight into the scores.
• Internal inconsistencies in HumDial. The text says the dataset has eight scenarios, but its table and description list nine. The text also says Gemini-2.5 has the lowest latency at 1.301 seconds, yet four entries in the table list lower delay.
• Narrow model coverage. M3-DuplexBench tests four Moshi-based models. Its Moshi results also use 5 seconds of prepended silence in the None and User settings, which affects latency comparisons.
• Scope of "Take the Floor". It is a five-page, English-only study with synthetic monologues. It does not say whether models fail to notice errors or notice and stay quiet.
• OmniVoice claims. "State of the art" holds on some metrics and not others, and misuse risk from high speaker similarity is acknowledged by the authors but not quantified.