Full-Duplex Voice AI Explained: Why Handling Interruptions Is the Hardest Problem

Handling interruptions is hard because a full-duplex voice agent has to make three separate decisions in a fraction of a second: is this speech meant for me, should I stop, and what should I say next. In the EchoChain benchmark (arXiv 2604.16456, 8 April 2026), no model exceeded a 50% pass rate on interrupted tasks, and the best of four reached 48.5%. Earlier benchmarks show the other half of the problem: models that yield fastest also stop for speech that was never addressed to them.

All results below are the authors' own, mostly on synthetic audio, and have not been independently replicated. Model versions are the ones each paper evaluated, not necessarily what ships as of 15 October 2026.

What is full-duplex voice AI, and how is it different from a normal voice assistant?

Full-duplex voice AI listens and speaks at the same time. Full-Duplex-Bench (arXiv 2503.04721) contrasts it with half-duplex systems, which work turn by turn, processing one audio stream at a time and switching roles when a turn-changing signal arrives.

The older pipeline is voice activity detection, speech recognition, a text LLM, then text-to-speech. The Moshi paper (Kyutai, arXiv 2410.00037, 17 September 2024) says this design adds latency of several seconds, drops non-linguistic information, and depends on speaker turns that ignore overlapping speech, interruptions and interjections. Those are the authors' framing, but the turn-based assumption is the common thread across the papers below.

How do full-duplex models actually work?

They model the user's audio and their own audio as parallel or interleaved streams, so there is no hard "your turn, my turn" switch. Three designs show the range.

SyncLLM's authors also point to a human benchmark: people can take turns with gaps as small as 200ms while language generation takes around 600ms, so listeners must anticipate what the other person will say. That is why SyncLLM predicts both speakers' speech units ahead in chunks of 160 to 240 ms, which the authors say keeps it resilient to Internet latencies of up to 240 ms.

How do production APIs handle an interruption?

They usually bolt interruption handling onto the audio stream rather than modelling it end to end. OpenAI's Realtime API guide says that with VAD enabled the service detects user speech, cancels the ongoing response and starts a new one.

The model also needs to know where it was cut off. OpenAI's guide calls this truncation: removing the unplayed portion of the last response from the conversation. On WebRTC and SIP connections the server does this automatically. Over WebSocket, the client must stop playback and send a conversation.item.truncate event.

Deciding when the user is finished is a related problem. OpenAI's VAD guide describes a semantic VAD mode that uses a classifier on the words spoken, so trailing-off audio gets a longer timeout than a definitive statement. An "eagerness" setting (low, medium, high, auto) controls how eager the model is to interrupt the user.

How do researchers measure interruption handling?

They test overlap in controlled scenarios and score what the model does next. Four benchmarks, all from 2025 and 2026, each isolate a different layer of the problem.

Does a model that stops fast also handle interruptions well?

No. In Full-Duplex-Bench v1.5 (arXiv 2507.23159, submitted 30 July 2025, revised 26 April 2026), the authors report a trade-off between responsiveness and holding the floor. Their five systems split into a "responsive" group and a "floor-holding" group.

On a true interruption, the paper measures how often the model answers the new speech (RESPOND, higher is better), how often it ignores it and carries on (RESUME), and how long it takes to stop (stop latency).

The fastest yielder pays for it elsewhere. On speech aimed at someone else, GPT-4o Realtime responded in 0.91 of trials, and on background speech in 0.93. Gemini resumed its own speech in 0.99 of the talking-to-others trials, and Nova Sonic resumed in 0.98 of the background-speech trials, yet both were slow on real interruptions. The authors write that human-like repair typically needs a response latency of 1.5 s or less, which only GPT-4o Realtime met, "but often with the wrong action".

Can a model answer the interruption and still get the task wrong?

Yes, and this is the failure that timing metrics miss. EchoChain (Modi, Mahajan, Wetter and Welles of Labelbox) injects a barge-in at a standardized point after the assistant starts speaking, then checks whether the continuation reflects the new information.

The paper names three failure patterns:

• Contextual inertia: the model acknowledges the interruption but keeps generating as if nothing changed.
• Interruption amnesia: the update is incorporated, then overwritten as generation continues.
• Objective displacement: the model answers only the interruption and abandons the original task.

Across 200 interrupted conversations per model, the mean pass rate (every rubric criterion must be met) was:

The authors ran a paired control to check that the interruption itself causes the damage. On 48 sampled conversations across the four models, they found 92 failures when interrupted and 55 when the same content arrived after a complete response. That is a 40.22% reduction, which the authors read as evidence that many errors come from state revision under interruption, not from task difficulty alone. Note that 55 failures remained even without interruption.

Failure profiles differed. Per the paper's counts, Nova Sonic 2 had 73 contextual-inertia failures out of 147, while Gemini Live had 65 amnesia and 64 displacement failures out of 167.

Can models adapt mid-sentence the way people do?

One early study of a single model suggests not yet. A September 2026 paper from Besimple AI (arXiv 2609.13117, submitted 11 September 2026) argues that "keep talking or stop" is a false binary. People often keep speaking while folding in what the listener just said, such as a supplied word or a correction. The authors call this in-turn adaptation.

Their case study used 300 human-confirmed cues from unscripted English conversations and retained 208 pairs. On the 66 collaborative pairs, recorded speakers adapted in 68.2% of cases, versus 34.8% for PersonaPlex. The model otherwise continued unchanged (42.4%) or yielded (22.7%). It tests a single model.

A second paper, Take the Floor When Asked, Not When Needed (arXiv 2609.19596, 17 September 2026), looks at the reverse: whether a model speaks up on its own. The authors report that Moshi and PersonaPlex assign lower text-token probabilities to false facts than to neutral statements in the first 2 seconds after a trigger, and that pauses or permission to interrupt do not close the gap. Stimuli and code are public at github.com/vocaliodmiku/take-the-floor.

Does the evidence hold up?

It is useful but thin in places. Here are the main limits, with some analysis of my own marked as such.

• Synthetic interruptions. v1 and v1.5 build interruptions with text-to-speech, and EchoChain uses voice cloning. Full-Duplex-Bench v1 says its analysis is limited to English and does not tie behaviours to human preferences.
• LLM judges. v1.5 uses GPT-4o to label behaviour, and v2 reports Pearson correlation of 0.59 to 0.69 between its LLM judge and humans. The v2 authors list prompt sensitivity and model bias as risks.
• Dated endpoints. v1.5 tested 2024 and 2025 API versions. The EchoChain models are also named versions (for example GPT-realtime-2025-08-28), so its results describe those versions only.
• Industry-team authors. EchoChain's authors list Labelbox and Duplex Cue's authors list Besimple AI as their affiliation, not university labs. That does not make the results wrong, but it is context when judging independence.
• Small cells. Duplex Cue's key comparison rests on 66 pairs, so one pair moves a percentage by about 1.5 points (my arithmetic). Grok's 48.5% and GPT-realtime's 45.0% in EchoChain differ by 7 of 200 conversations, so the top two are not clearly separated (my reading). For the same reason, see why small trial counts matter in robot evaluation.

Reading note: the full PDFs of Full-Duplex-Bench v1, v1.5, v2, EchoChain and SyncLLM were read. For Moshi, Duplex Cue and Take the Floor, this post relies on the abstracts and excerpts, so their headline numbers are the authors' claims as stated there.

What does a working researcher or engineer take from this?

Evaluate interruption as several separate skills, not one score. My analysis, based on the papers above:

1. Report stop latency and what happens after. v1.5's split between RESPOND and RESUME shows that a fast stop can still be the wrong action.
2. Test non-addressee speech. Background speech and side talk are where the fastest yielder failed in v1.5.
3. Check task state after the barge-in. EchoChain's rubric catches errors that fluent audio hides.
4. Run a no-interruption control. EchoChain's paired design separates interruption damage from task difficulty.
5. Go multi-turn. Full-Duplex-Bench-v2 reports that performance degrades as dialogues continue. Under Fast pacing, GPT-Realtime scored 4.02 on Correction and 4.51 on Entity Tracking (1 to 5 scale), while Moshi scored 2.88 and 2.76 and Freeze-Omni 2.74 and 2.62.

For architecture choices, v1 offers a hint. The authors attribute Freeze-Omni's lower takeover rate during pauses (0.481 on Candor, versus 0.980 for Moshi) to its explicit turn-taking control module, and suggest such a module could help end-to-end systems too.

What's still unclear?

Several questions remain open, and some sources conflict on framing.

The v1 and v1.5 pictures differ. In v1, Moshi has a takeover rate of 1.000 on interruptions and Gemini Live 0.891 (higher is better there). In v1.5, Moshi answers the interruption in 50% of trials and Gemini in 33%. The likely cause, in my reading of the definitions, is that v1 counts any non-silent, non-backchannel speech as a takeover, so a model that talks over the interruption and carries on its old answer still scores as having taken the turn. v1.5 separates that case as RESUME. The two papers share a lead author and group, so this is a refinement of the metric rather than a dispute.

The latency numbers are not comparable across papers. v1 reports Moshi's latency after interruption as 0.257 s, while v1.5 reports 1.16 s stop latency and 1.47 s response latency. They measure different intervals on different protocols.

No paper shows current production systems. The benchmark endpoints are dated snapshots. Whether newer releases close the gaps is not established in these sources.

Language coverage. Every benchmark here is English-only, and the v2 authors note that other languages involve different overlap patterns and cultural norms for timing and backchannels.

What good looks like. The v1 authors say users must decide which behaviours are desirable for their goals. There is no agreed target for how often a voice agent should yield, resume or adapt.