Meta's "Sharpening Tax": Post-Training May Be Making AI Worse at Exploring (2610.01509)

Sharpening Tax in Post-Training (arXiv 2610.01509, submitted 1 October 2026) reports that post-trained models win at pass@1 but often lose to their base models at pass@K. On WebShop with Gemma-4-31B, the authors report over 85% pass@128 for the base model versus 56% for the post-trained one. The paper comes from Changdae Oh and nine co-authors, most listed at Meta Superintelligence Labs, with University of Wisconsin-Madison and Stanford. All numbers below are the authors' own, not independently replicated.

We read: the full paper text (arXiv HTML version, including appendices), not only the abstract.

What problem does this paper solve?

It asks whether reinforcement learning (RL) post-training gives a model new agentic capabilities or only sharpens what the base model already does. Earlier evidence for "sharpening" came mostly from math and coding, where pre-training data is abundant and final-answer checks can reward a lucky guess.

The authors argue agentic tasks are a harder test. Multi-turn tool use is rare in pre-training data, so you would expect post-training to be necessary for success at all. They check this on three benchmarks where success depends on the environment state the model reaches, not on matching an answer string.

The question has a longer history. Yue et al. (arXiv 2504.13837) concluded that RLVR methods improve sampling efficiency but rarely elicit novel reasoning beyond the base model. Sharpening Tax extends that line of argument to agents and adds a metric to measure it.

How does the Sharpening Tax work?

The Sharpening Tax is the gap in "test-time scalability" between a base model and its post-trained version, summed over the whole sampling budget. The paper defines raw scalability as A(K) = sum over k from 1 to K-1 of [pass@K - pass@k]. It then normalises by the single-shot failure rate to get S(K) = A(K) / ((K-1)(1 - pass@1)).

The tax is Tax(K) = X_Base(K) - X_Post(K). A positive value means post-training reduced how much the model gains from extra rollouts.

In plain terms: pass@1 is the chance one attempt succeeds. pass@K is the chance at least one of K attempts succeeds. A model that is great on the first try but never solves hard tasks has a flat pass@K curve, so it pays a high tax.

What did the authors actually run?

They compared 14 open base and post-trained checkpoint pairs on three agentic benchmarks. The setup is summarised below, per the paper's appendix.

Base models have no chat template or tool-call format, so the authors built a "harness": a simple system prompt plus relaxed tool-calling and parsing. Base models get this harness; post-trained models use each benchmark's default scaffolding. The authors say the same harness hurts post-trained models.

The authors also note the post-training recipes of these open checkpoints are not fully disclosed. They use "RL post-trained" as shorthand, even though the models may combine SFT, preference optimisation and RL in unknown proportions.

What are the results?

Post-trained models lead at small K. Base models scale more steeply and cross over in most settings as K approaches 128.

The mechanism is the middle rows. Post-training pushes tasks to two extremes: always solved or never solved. The authors call this bimodalization. More tasks become reliable, but more also become hopeless, and extra rollouts can no longer rescue the hopeless ones.

Scale matters. On WebShop, the crossover budget for Gemma-4 falls from above 128 rollouts at 4B to about 3 at 31B. For small models, the tax is often negative at K of 8 or less, so sharpening helps when compute is tight.

The harness itself does a lot of work. Averaged over eight base models, per the paper, BFCL pass@32 rises from 15.19 to 49.13 and WebShop pass@32 from 39.48 to 49.15 with the harness. ACEBench is roughly unchanged (89.25 without, 87.69 with).

Does PTGS fix the problem?

PTGS (posterior-tempered group sampling) is the authors' mitigation. It tracks each prompt's success rate with a discounted Beta posterior, draws a Thompson sample, and sets the sampling temperature as T = tau^h(p_hat). Hard prompts are heated toward tau, easy ones cooled toward 1/tau. It is used only during RL training and changes nothing else in the update, the authors say.

They fine-tuned Qwen2.5-7B-Instruct on Sokoban and FrozenLake with PPO and GRPO, 200 steps, five runs each.

PTGS lowers the tax in every row and raises both pass@1 and pass@128. Even so, task-specific RL still loses coverage relative to the base model: pass@128 stays below the base model's in all four PTGS rows, and FrozenLake GRPO barely moves (0.029 to 0.028). The authors also report that PPO's entropy falls steadily while PTGS keeps it higher, which they link to preserved exploration.

Does the evidence hold up?

The core pattern looks consistent, with caveats that matter. These are the weak points, in rough order of importance.

1. The comparison is not symmetric. Base models get a custom harness, post-trained models get default scaffolding. The paper says the harness improves base-model scores on benchmarks they struggle with, and hurts post-trained ones. A harness tuned for the post-trained side could shift the crossover point.
2. The tax is observational. The authors state that pre-training and post-training data are undisclosed, so they cannot say what causes sharpening. They did not compare SFT, RL and distillation.
3. Pass@128 is expensive. Reading coverage off 128 samples needs 128 times the compute and a reliable verifier. The paper's own benchmarks supply one; many real tasks do not.
4. The PTGS test is small. One 7B model, two game environments, five runs, and for GRPO the temperature spread was picked on peak validation success per environment. That is a long way from frontier agent training.
5. Narrow scope. Three benchmarks, text-only agents. The authors list multimodal and vision-language-action models as open.
6. Tax_S is normalised. The calibrated variant can rise or fall with budget, so one number hides shape. The paper reports both the raw and calibrated variants.

In the paper's favour: it fixes one BFCL category and reports why, uses process-checked success rather than final-answer matching, reports 95% bootstrap confidence intervals, and includes a thinking-mode ablation showing the trend holds. With thinking mode on, post-trained models improve but base models still catch up at large K.

Code is public at github.com/changdaeoh/sharpening-tax, with a project page. The full sweep covers 28 checkpoints at up to 128 rollouts per task, so reproducing it is not cheap.

How does it compare with related work?

Sharpening Tax adds two things: a single metric that can be estimated from few rollouts, and evidence outside math and code. It does not overturn earlier work. For background on how RL-based post-training became standard, see our explainers on DeepSeek-R1 and InstructGPT and RLHF.

What does a working researcher take from this?

Report coverage next to accuracy. The authors suggest reporting Sharpening Tax alongside accuracy. If your pipeline samples many candidates and verifies them, a base model with a harness may be a better starting point than the instruct checkpoint.

Practical points:

• Check the crossover. If your budget is above the crossover K for your model size, the base model may cover more tasks. Per the paper, larger models cross sooner.
• Estimate cheaply. The authors report that 8 rollouts on half the tasks predict the 32-rollout tax with Spearman 0.85.
• Do not raise temperature globally. The paper says this improves pass@K but degrades pass@1, which is the problem PTGS tries to avoid.
• Match the use case. For a single reliable answer, sharpening is a benefit. For search, discovery or verified sampling, it costs coverage. For background on how model size is usually analysed, see our explainer on scaling laws.

What's still unclear?

• Cause. Without known data mixes, the paper cannot separate the effects of RL, SFT and distillation. Controlled training is the authors' stated next step.
• Fairness of the harness. No result shows a post-trained model given a harness tuned for it.
• Beyond text agents. Whether the trade-off holds for multimodal models and robot policies is open, per the authors.
• Frontier models. The 14 pairs top out at 35B parameters. The paper does not test closed frontier systems.
• Does PTGS scale? The evidence is two toy environments on one 7B model.
• The Meta framing. The authors are largely Meta Superintelligence Labs staff, but the models studied are Gemma-4, Ministral-3 and Qwen checkpoints. The paper makes no claim about Meta's own models.
• Title framing. "Worse at exploring" is a fair summary of the coverage loss, but the same models are more reliable. The authors present this as a trade-off, not a net loss.