"The Teacher Is a Direction, Not a Destination": RIDE's Bold Distillation Claim (2609.36484)

RIDE, from Hao Li, MeiJia Chen, Weijie Ren and co-authors, is a distillation method that tries to take a student past its RL-trained teacher. In the paper (arXiv:2609.36484, 29 September 2026), the authors report that RIDE reaches 56.38 mean Avg@16 on a 1.5B math pair against the teacher's 55.30, and that its mean sits above the teacher's on all four model pairs tested, by 0.32 to 1.08 points. Those are the authors' numbers on three small benchmarks, and the paper itself concedes the margin is within one seed standard deviation on three of the four pairs.

We read: the full PDF (19 pages), including appendices.

What problem does RIDE try to solve?

It tries to let a student keep improving past the teacher instead of stopping at imitation. In standard on-policy distillation (OPD), the student samples its own responses and the teacher scores every token, so the teacher is the endpoint (Thinking Machines Lab, 27 October 2025).

Prior work already tried to go further. ExOPD (arXiv:2602.12125, February 2026) scales the implicit reward, the log-probability ratio between teacher and a reference model, by a factor above 1. Its authors report that this lets a student surpass domain-expert teachers when the experts were all trained from the same student.

The RIDE authors argue this runs into two problems:

• The language-model head blunts the signal. They report that the 512 weakest directions of the head carry 79.8% of the teacher-to-base residual's hidden-state energy but only 30.0% of its centered-logit energy.
• Sampled log-ratios are noisy, and extrapolation amplifies the noise. Output-space extrapolation uses the log-ratio of one sampled token, and the noise term is scaled by (lambda - 1) squared.

How does RIDE work?

RIDE moves the extrapolation from output probabilities to hidden states. The student starts at the teacher's pre-RL checkpoint. On each student-generated prefix, the frozen teacher and the frozen base model both run, and their layerwise hidden-state difference is the "RL-induced residual."

The paper's target (Eq. 6) and loss (Eq. 7, simplified) are:

delta[l,t]  = h_teacher[l,t] - h_base[l,t]              # RL-induced residual
target[l,t] = h_teacher[l,t] + (lambda - 1) * delta[l,t]
loss        = mean over layers l, positions t of
              (1/d) * || h_student[l,t] - stopgrad(target[l,t]) ||^2

At lambda = 1 the target is the teacher and the method reduces to OPRD, which matches hidden states without extrapolating. At lambda = 1.25, the default, the target sits a quarter of the residual beyond the teacher. The authors also show the regression equals maximizing a linear "directional reward" under a quadratic penalty that keeps the student near the teacher (Proposition 4.2).

Setup details from the paper's Table 5:

• Training prompts come from DAPO-Math-17K, for 500 steps, with 8 prompts per step and 2 responses per prompt.
• The optimizer is AdamW at a peak learning rate of 1e-5, on 4 H200 GPUs.
• RIDE supervises all layers and the last 2,000 response positions.
• Evaluation uses Avg@16 at temperature 0.70.

Compared with OPRD, RIDE adds one frozen forward pass of the base model on the teacher worker, and the authors say it adds no extra communication.

What are the results?

The authors report RIDE's mean ahead of OPRD and ExOPD on every pair, and ahead of the RL teacher on all four. All figures below are Avg (the unweighted mean of Avg@16 on AIME24, AIME25 and AIMO), averaged over three training seeds for trained methods, from the paper's Table 1.

Evaluation size: AIME24 has 30 problems, AIME25 has 30, and AIMO (AMC 2022 to 2023) has 83, each sampled 16 times. The "RIDE minus teacher" column is simple subtraction from the paper's table. The SD column is from the paper's Table 4.

The authors also report that RIDE's margin over OPRD runs from 0.97 to 4.06 points, and that it beats ExOPD by 9.1 points on average. ExOPD falls below its teacher on every pair, and on Qwen3-4B it ends 14.1 points under the untouched student.

The base model behind the 1.5B pair is DeepSeek-R1-Distill-Qwen-1.5B, and its RL teacher is JustRL-DeepSeek-1.5B.

Does the evidence hold up?

Partly. The comparison against ExOPD and OPRD looks strong. The headline claim of beating the teacher is much thinner.

Margins over the teacher are within noise on most pairs. The paper says that on three pairs the margin is within one across-seed standard deviation. Only the R1-Distill pair, at +1.08 against an SD of 0.97, clears that bar, and barely. The teacher's own score is a single fixed-checkpoint evaluation, so its sampling variation is not captured in any SD.

The benchmarks are small. With 30 problems on each AIME set, one problem is worth 3.3 points of accuracy on that benchmark. On Llama-3.2-3B the paper notes the AIME scores sit near the evaluation floor (teacher AIME25: 0.42), so "the evidence there rests mainly on AIMO."

The direction ablation is weaker than the title suggests. The most important test is whether the RL-induced direction matters at all. On the R1-Distill pair, the paper compares RIDE with controls that keep displacement size fixed but change its direction:

A random direction of the same norm scores 55.04, only 1.34 points under RIDE, and RIDE's seed SD on this pair is 0.97. Table 3 reports no seeds or SDs for the controls. The gap supports the authors' reading but does not settle it with this sample.

The coefficient was tuned on the reporting benchmarks. The lambda sweep (Table 2) uses one run per coefficient, and its columns are the same three benchmarks as the main table. RIDE scores 55.2, 55.4 and 55.3 at lambda 1.15, 1.25 and 1.35, a flat top with differences far smaller than seed noise. The default of 1.25 is a reasonable choice, but its tuning and testing overlap.

Three of four teachers were trained by the authors. Only JustRL-DeepSeek-1.5B was an existing public teacher. Just-Qwen3-4B, Just-Llama-3.2-3B and Just-Phi-4-mini were obtained by applying the JustRL recipe to the public bases. For these three, the paper says RL moved the teacher only 2.8 to 6.0 points from its base, against 16.3 points for the R1-Distill pair.

How does RIDE compare with related work?

RIDE combines two ideas from earlier 2026 papers: hidden-state matching and reward extrapolation. The table lays out where each method places its supervision and target.

OPRD (arXiv:2606.06021, submitted 4 June 2026) aligns representations on the same rollouts and bypasses the head, which is the part RIDE builds on. RIDE's variance claim is a theory result with a measurement: the paper reports that ExOPD's conditional variance rises 12.6 times from lambda = 1 to lambda = 2 on 9,152 validation prefixes, while the RIDE per-position gradient does not depend on which next token is sampled.

One conflict to note is between settings, not necessarily between findings. The ExOPD authors report their method surpassing domain teachers in their own experiments. The RIDE paper finds ExOPD below its teacher on every pair in its setup, using the same lambda and pre-RL reference. The RIDE authors ran ExOPD themselves, so an independent rerun would help.

What can a working researcher take from this?

Treat RIDE as a promising and cheap-to-try idea, not an established result. Four takeaways follow from the paper:

1. Hidden-state supervision gives a lower-noise signal than sampled log-ratios. The OPRD-style deterministic gradient is the more solid half of the paper's argument, and OPRD at lambda = 1 lands within 1 point of the teacher on two of the four pairs.
2. Extrapolating beyond a teacher is risky in output space. ExOPD degraded at every lambda above 1 in the authors' sweep, from 52.9 at lambda 1 to 46.2 at lambda 2. If you extrapolate, the paper's evidence favors doing it before the head.
3. You need the pre-RL checkpoint. If you only have a released RL model without its base weights, the method does not apply.
4. Check calibration and safety yourself. The authors say the student is trained on hidden-state targets that no model has actually produced, so its calibration and safety behavior "should be evaluated independently before deployment."

What's still unclear?

• Is the gain over the teacher real? The authors' own seed SDs (0.88 to 1.13) are as large as or larger than the margins on three of four pairs. More seeds and more problems would settle it.
• Does the RL-induced direction itself matter? The random-direction control lands within 1.34 points of RIDE. Seeds for the controls are not reported.
• Does it transfer beyond math? The authors state RIDE was evaluated only on mathematical reasoning with one RL recipe, and that it uses a single global lambda.
• Can others reproduce it? The paper says code, teacher checkpoints and evaluation scripts will be released upon publication. The project repository held a README and an assets folder as of 1 October 2026.
• Does amplification carry risks? The paper's ethics statement notes that because RIDE displaces the target beyond the teacher, biases or errors that RL introduced could in principle be amplified rather than merely copied.

All results here are the authors' claims, as of 16 October 2026.