Ant Group's Realtime-Venus vs GPT-4o and Gemini Live on Full-Duplex Voice: Where It Wins and Where It Doesn't (2609.13814)

In "Realtime-Venus: A full-duplex interaction system with asynchronous delegation" (arXiv 2609.13814, Venus Team at Ant Group and Tsinghua University, version 3 dated 23 September 2026), the authors report that their 9B Realtime-Venus-Audio model keeps talking through backchannels, other-directed speech and background speech more reliably than GPT-4o and Gemini 3.1 Live. Its continuation rates on Full-Duplex-Bench v1.5 are 0.97, 0.88 and 0.86 across those three scenarios.

The same table shows the model responds to 75% of real user interruptions, which is lower than GPT-4o (0.78), Gemini 3.1 Live (0.77) and Joy-Duplex (0.88). So "beats GPT-4o and Gemini Live" is true for three metrics and false for the fourth. All numbers below are the authors' own, not independently replicated.

We read: full PDF (29 pages, v3), plus the abstracts of the two benchmark and baseline papers it borrows numbers from.

What problem does this paper solve?

It targets two timing problems in voice assistants: deciding when overlapping speech is a real interruption, and keeping a conversation alive while a slow task runs. A full-duplex model keeps listening while it speaks. The hard part is classifying what it hears: a "yes" that means "go on", a remark to someone else in the room, background chatter, or a correction that should stop the answer.

The second problem is that some requests need tool calls or reasoning that outlast one conversational turn. The paper's design splits the two jobs without splitting the conversation. A fast "interaction loop" handles perception and speech, and a slower "capability loop" runs delegated work and returns results to the same session.

How does Realtime-Venus work?

Both models are adaptations of MiniCPM-o 4.5, which the paper calls an open-source 9B model. The architecture uses a Qwen3-8B language backbone, Whisper-Medium for audio and SigLIP2 for vision (the Audio variant drops the visual branch). The language model's output conditions discrete S3 speech tokens, and a streaming flow-matching decoder turns them into waveform.

Streaming runs in one-second units. At each unit the model predicts a <|listen|> or <|speak|> token, and <|turn_eos|> closes a turn. The paper describes three behaviours for overlapping speech:

• Backchannel ("yes", "right"): the model keeps its <|speak|> state and continues.
• Interruption (a correction, redirect or new request): the model emits <|turn_eos|>, drops the stale unspoken text and incorporates the new intent.
• Pauses and background noise: the model stays in <|listen|> and does not start or end a response.

Delegation uses four added tokens, <delegate>, </delegate>, <backend> and </backend>. The model writes a private natural-language task inside a delegate span, hidden from the user. Realtime-Venus-Harness freezes the evidence available at that moment, routes the task to a registered capability, polishes the result into a spoken-form reply and sends it back as private context. The frontend decides when to speak it. Work items move through QUEUED, RUNNING, COMPLETED, DELIVERING and DELIVERED, and stale results are excluded.

Training data and setup

The shared post-training corpus has over 2.8 million samples in nine categories, according to the paper. For the Audio model, the relevant slices are 470k general audio understanding, 205k spoken question answering, 100k speech-only duplex and 90k audio delegation samples. Sessions are planned from scenario seeds, converted to speech with added hesitations, laughter and background audio, and aligned on a shared clock. The corpus covers Chinese, English and code-switching.

Only the language model ("Thinker") is updated. The acoustic decoder stays fixed.

What are the results on full-duplex voice benchmarks?

On Full-Duplex-Bench v1.5, Realtime-Venus-Audio has the highest continuation rate in the backchannel and background-speech scenarios and second highest for speech directed at others, behind Realtime-Venus-Omni. The benchmark has four scenarios. For interruptions the desired behaviour is to respond (C_RESPOND, higher is better). In the other three, continuing the previous answer (C_RESUME) is preferred.

Table 1 reproduces the paper's Table 6 for selected models (Table 6 in the paper also lists Freeze-Omni and Moshi, and reports the opposite metric in each scenario).

Source: arXiv 2609.13814, Table 6, as reported by the authors. Data as published 23 September 2026.

Three things stand out:

• Against GPT-4o: the gap on non-interruptive speech is large. GPT-4o resumes only 0.02 of the time when speech is directed at others, and 0.04 for background speech, meaning it usually answers speech that was not meant for it.
• Against Gemini 3.1 Live: the margin is 0.02 on backchannels, 0.22 on other-directed speech and 0.20 on background speech.
• Against the base model: Realtime-Venus-Audio raises interruption response from 0.60 to 0.75 over MiniCPM-o 4.5, and also improves the other three continuation rates.

The authors say this themselves: strong continuation does not necessarily imply strong interruption handling.

Other reported results

Source: arXiv 2609.13814, Tables 4 and 5. Qwen3-Omni is a 30B-A3B model. Margins are often a few tenths of a point.

On tool use, the picture is less flattering. On the tool-use part of Full-Duplex-Bench v3, GPT-Realtime leads all three metrics at 87.6% tool-selection F1, 68.0% argument accuracy and 60.0% Pass@1. Realtime-Venus-Omni scores 86.0%, 53.1% and 43.0%. Realtime-Venus-Audio scores 82.0%, 52.2% and 42.0%. Gemini 3.1 Live scores 54.0% Pass@1, above both. The paper also reports that Realtime-Venus-Omni leads online models on six of eight video benchmarks, including StreamingBench (70.2%) and OVO-Bench (64.7%), but this post focuses on voice.

Does the evidence hold up?

Partly. The continuation numbers are real table entries, but several features of the setup limit how far they generalise.

1. Borrowed baselines. The paper's Table 6 notes that GPT-4o numbers come from the original Full-Duplex-Bench v1.5 paper, and Joy-Duplex and Gemini 3.1 Live numbers come from the JoyAI-Talker paper. The Venus team did not re-run them. API models change over time, and the prompts and audio settings used elsewhere may differ from the Venus team's runs.
2. No uncertainty reporting. The results tables give single values with no confidence intervals, seeds or run counts. Differences of 0.01 to 0.03 (for example 0.97 versus 0.95 on backchannels, or 0.75 versus 0.77 on interruptions) cannot be separated from noise on this evidence.
3. No ablations. The paper states that its results do not isolate the contribution of individual training components. The gain over MiniCPM-o 4.5 cannot be attributed to the duplex data, the delegation data or the training recipe.
4. Synthetic, bilingual training data. The duplex corpus is planned from scenario seeds and realised as speech, in Chinese, English and code-switching. Whether that transfers to messy real calls is not tested here.
5. In-house delegation benchmark. The delegate benchmark is internally constructed. On it, Realtime-Venus-Audio has 92.22% delegation recall but only 39.44% non-delegation specificity, so it frequently delegates routine requests. Realtime-Venus-Omni shows the reverse: 84.44% specificity and 68.33% recall.
6. Scope. The paper says its aggregate scores do not establish broader reasoning capability, and that ambiguous overlaps and response-transition timing need further evaluation.

On reproducibility, the model card lists an Apache 2.0 licence, and the project page links GitHub code, a Hugging Face model, ModelScope and a beta Android app. That lets others re-run the benchmark on the same checkpoint, which is the check this paper most needs.

How does it compare with related work?

Realtime-Venus sits between three lines of work, and the paper positions it against each.

The v1.5 benchmark's own framing helps read the result. Responsive models answer quickly to any input, while floor-holding models filter overlapping events. Realtime-Venus-Audio looks closer to the floor-holding end on these numbers, with some interruption sensitivity retained. A model that never interrupts would score perfectly on three scenarios and zero on the fourth, which is why the authors report the four scenarios separately and decline to aggregate them without an explicit weighting.

What does a working researcher take from this?

Read Table 6 per scenario, not as one score. If your product is a voice agent in a noisy room or a meeting, continuation and false-trigger rates matter most, and the Realtime-Venus numbers are strong there. If it is a customer-service agent where users correct the bot often, interruption response and tool-call accuracy matter more, and this paper's numbers are weaker.

Other practical points:

• The delegation pattern is reusable. Freezing evidence at request time, checking freshness before reintegration and tracking playback acknowledgments are design choices you can copy regardless of the model.
• Check tool-use results separately. Selecting the right tool is not the same as completing the task. Realtime-Venus-Omni reaches 86.0% tool-selection F1 but 43.0% Pass@1, which requires correct arguments and no extra calls.
• Re-run before you rely on it. With the checkpoint public, a same-setup comparison against current GPT-4o and Gemini Live endpoints is the missing experiment.

What's still unclear?

Several questions remain open.

• Same-setup comparison. Would the continuation advantage hold if all models were re-run under one protocol on the same date? The paper copies several baselines from other sources.
• Interruption latency. The paper says response-transition timing needs further evaluation. A 75% response rate says nothing about how fast the model stops talking.
• Source of the gains. Without ablations, the contribution of the duplex data versus the delegation data versus the base model is unknown.
• Delegation quality. The delegate benchmark measures routing decisions only. The authors state that successful execution and integration of results need additional evaluation.
• Real-world transfer. Training sessions are synthetic. Natural overlapping speech, accents and noisy channels are untested in the reported results.
• Evaluation size. The results tables do not report how many audio samples sit behind each rate, so the precision of each figure is unknown.

The paper's own future-work list names finer-grained streaming chunks, longer context and more complex multi-step tool tasks, including recovery from delayed or failed tool calls.