Qwen3 Technical Report — L2 #

1. TL;DR #

Qwen3 fuses a reasoning model and a chat model into one checkpoint via a 4-stage post-training pipeline (Long-CoT cold start → Reasoning RL → Thinking-Mode Fusion → General RL), adds a user-controllable thinking budget, and builds its 6 small models by on-policy distillation that beats RL at ~1/10 the GPU hours.

2. Q1/Q2/Q3 #

Q1 — 痛点 (problem). Frontier deployments needed two separate model families: a chat-optimized model (fast, GPT-4o-style) and a dedicated reasoning model (slow, deliberate, QwQ/o1-style). Users had to switch endpoints per query, doubling serving cost and eliminating any fine-grained control over how much reasoning effort a single query gets. Separately, training a full family of small models from scratch with the same reasoning quality as the flagship is prohibitively expensive.

Q2 — 方法 (method). A single model is trained to operate in both thinking mode (emits a ... block before answering) and non-thinking mode (empty think block, immediate answer), selectable at inference via /think / /no_think chat-template flags. The flagship (Qwen3-235B-A22B, an MoE) is trained through four sequential post-training stages; the two "thinking" stages install reasoning, the two "non-thinking" stages fuse in chat behavior without destroying it. Six lightweight models are then produced by strong-to-weak distillation rather than repeating the 4-stage pipeline.

The 核心技术壁垒 (the single hardest thing to replicate): making thinking and non-thinking coexist in one weight set and letting the user smoothly interpolate between them via a token budget — where the intermediate "answer-on-incomplete-thought" behavior emerges without being explicitly trained. The whole value proposition (one model, dial-a-reasoning-budget) hinges on this emergent property surviving the fusion stage; getting the stage ordering and data mix right so that fusion does not catastrophically degrade the RL-installed reasoning is the moat.

Q3 — 结果 (result). The flagship hits AIME'24 85.7, AIME'25 81.5, LiveCodeBench v5 70.7, CodeForces 2056, BFCL v3 70.8 in thinking mode — competitive with DeepSeek-R1 (671B/37B-activated) and closed models, at 235B total / 22B activated. On-policy distillation lifts Qwen3-8B far above RL (AIME'24 67.6→74.4) while using 1,800 vs 17,920 GPU-hours. Base models reach comparable quality to Qwen2.5 models 1 tier larger; MoE base matches dense base at ~1/5 activated params.

3. 架构 / 方法图 #

Paper's Figure 1 (p9), verbatim (caption: "Post-training pipeline of the Qwen3 series models.").

Figure 1 is the load-bearing diagram of the whole recipe. The flagship path runs left-to-right through four stages: §4.1 Long-CoT Cold Start (SFT to install reasoning format) → §4.2 Reasoning RL (GRPO on hard verifiable queries) → §4.3 Thinking Mode Fusion (continual SFT that grafts non-thinking behavior onto the RL model) → §4.4 General RL (broad reward system over 20+ tasks). A parallel Strong-to-Weak Distillation path (off-policy → on-policy) branches off to produce the 5 dense + 1 MoE lightweight models, bypassing the expensive 4-stage pipeline entirely. The reader should notice the asymmetry: the flagship earns its capabilities through RL, while the small models inherit them through distillation — this split is the report's core cost-saving thesis.

The pipeline before vs after versus the prior Qwen2.5+QwQ world:

	before (Qwen2.5 + QwQ)	after (Qwen3)
Reasoning vs chat	two separate checkpoints	one checkpoint, mode-switched
Effort control	none (fixed per model)	user-set thinking budget (tokens)
Small-model training	independent full training	distilled from flagship
Mode selection	pick a model	/think / /no_think flag

4. 作者证明 #

无形式化作者证明 — 仅实证. This is an engineering technical report with 0 numbered display equations; it offers no convergence theorem, sample-complexity bound, or formal loss decomposition. All claims are supported empirically (37 tables). The two symbolic objects that carry weight are stated but not analyzed:

• On-policy distillation objective: the student generates on-policy sequences, then is fine-tuned to align its logits with a teacher (Qwen3-32B or Qwen3-235B-A22B) by minimizing KL divergence, $\min_\theta \mathrm{KL}\!\left(p_{\text{teacher}} \,\|\, p_{\text{student}}^{\theta}\right)$ over student-sampled trajectories. No formal argument is given for why this expands (rather than merely mimics) the student's exploration space; the claim rests entirely on the observed pass@64 improvement in Table 21.
• Reasoning RL: GRPO (group-relative advantage, no value model) is used on $3{,}995$ query-verifier pairs. Query survival at cold-start is governed by Pass@$N$ (fraction of $N$ QwQ-32B generations passing verification); the six removal criteria are heuristic filters, not a proof.

Six minimum checks (applied as empirical-report sanity checks):

1. Objective stated? Yes for distillation (KL to teacher logits) and RL (GRPO); no closed-form loss written for the SFT stages. Partial.
2. Assumptions? Verifiability of queries (rule/test-case checkable) is the key implicit assumption for RL; distillation assumes teacher > student. Both stated in prose, not formalized.
3. Inputs/outputs of one step? Reasoning RL: (group of rollouts per query + scalar verifier reward) → policy update. On-policy distillation: (student rollout + teacher logits) → student update. Clear.
4. Guarantee type? None formal; only monotone-improvement observations (AIME'24 235B 70.1→85.1 over 170 steps).
5. Where would it break? RL depends on entropy staying stable ("controlling the model's entropy to increase steadily or remain stable"); the paper flags this as the fragile knob. Fusion degrades AIME'24/LiveCodeBench (admitted regression).
6. Tightness/optimality? Not claimed. Data mixture optimized empirically at instance-level via small proxy models; scaling laws used only to predict optimal LR/batch, not proven optimal.

5. 实验与数据 #

The base-model headline is that Qwen3-235B-A22B-Base beats far larger MoE baselines on 14/15 benchmarks with ~1/3 the total params of DeepSeek-V3.



Paper's Table 3. Reading across the last column, Qwen3-235B-A22B-Base is bold (highest) on nearly every row despite 235B total / 22B activated vs DeepSeek-V3's 671B/37B — the efficiency claim (comparable quality at ~1/2 the activated params, ~1/3 the total) is carried by this table. The only loss is INCLUDE (73.46 vs DeepSeek-V3 75.17).

The single most important ablation in the paper is Table 21: on-policy distillation vs RL from the same off-policy-distilled 8B checkpoint.



Paper's Table 21 (tab:distillation). On-policy distillation reaches AIME'24 74.4 / AIME'25 65.5 / LiveCodeBench 60.3 at 1,800 GPU-hours, while RL reaches only 67.6 / 55.5 / 52.9 at 17,920 GPU-hours — ~10× the compute for worse results. The subtle, load-bearing detail is the pass@64 column (in parentheses): distillation raises AIME'24 pass@64 to 93.3 and AIME'25 to 86.7, whereas RL leaves both unchanged (90.0 / 83.3). This is the empirical basis for the "distillation expands exploration, RL only exploits" claim.



Paper's Table 22 (tab:stage_ablation). This stage-by-stage ablation shows the trade-off the authors chose to accept: general/instruction/agent capabilities climb steeply (ToolUse +7.1 then +15.1; ThinkFollow 88.7→98.9; CounterFactQA +10.9), but hard reasoning regresses in thinking mode (AIME'24 83.8→81.4, LiveCodeBench 68.4→65.7). The reader should notice the deltas are positive almost everywhere except the two hardest reasoning rows — this is the cost of fusing versatility into a reasoning model.



Paper's Table 11 (tab:instruct-235A22). The flagship-thinking comparison against reasoning baselines. Qwen3-235B-A22B leads on agent/coding (BFCL v3 70.8, CodeForces 2056/98.2%, LiveCodeBench 70.7) and multilingual math (MT-AIME 80.8, PolyMath 54.7), but visibly loses on GPQA-Diamond (71.1 vs Gemini2.5-Pro 84.0) and INCLUDE (78.7 vs o1 84.6) — an honest picture, not a clean sweep.



Paper's Figure 2 (p20), verbatim (caption: "Performance of Qwen3-235B-A22B with respect to the thinking budget."). Accuracy rises smoothly and monotonically with the allocated thinking budget across four Math/Coding/STEM benchmarks. The point is that the budget knob is a usable dial — performance degrades gracefully as the budget shrinks, rather than collapsing — which is what makes the emergent early-halt behavior practically valuable.

6. 论证链 #

Step	Claim	Support (paper-internal)
1	A single model can hold both thinking and non-thinking modes if fusion is done after reasoning is installed by RL.	§4.3 Thinking Mode Fusion: continual SFT on the Reasoning-RL model; chat template with /think//no_think + empty think block for non-thinking.
2	Given (1), an intermediate "answer on incomplete thought" behavior appears for free, enabling a user thinking budget.	§4.3: early-halt by inserting a fixed stop-thinking string once thinking length hits the threshold; "not explicitly trained but emerges naturally."
3	Given (2), the budget dial yields smooth, monotone accuracy scaling.	§4.7 + Figure 2: accuracy rises smoothly with budget on 4 benchmarks.
4	Small models need not repeat the 4-stage pipeline; distilling flagship logits is enough.	§4.5 + §4.7: on-policy distillation Qwen3-8B beats RL at 1/10 GPU-hours (Table 21).
5	Distillation is not mere mimicry — it raises the exploration ceiling.	Table 21: distillation raises pass@64; RL does not.
6	The fusion in (1) costs some peak reasoning, but the authors accept it for versatility.	§4.7 + Table 22: thinking-mode AIME'24/LiveCodeBench decline after Stage 3/4; explicit "we choose to accept this trade-off."

7. 实现 cross-reference #

[实现未公开] — the training pipeline, data pipeline, and RL/distillation code are not released; only model weights (Apache 2.0) and the chat template are public. The one directly reproducible implementation artifact is the chat template that encodes the mode-switch mechanism, referenced by the paper at Qwen/Qwen3-32B/blob/main/tokenizer_config.json on Hugging Face — this file implements the block handling and the enable_thinking=False toggle (Table 9).

核心技术壁垒 (§7 dedicated paragraph). The hardest-to-replicate insight is not any single algorithm (GRPO, KL distillation, and cold-start SFT are all off-the-shelf) but the stage-ordering discipline that lets thinking-mode fusion graft chat behavior onto an RL-hardened reasoning model while (a) preserving mode-switch fidelity (ThinkFollow rises to 98.9) and (b) letting the graded early-halt budget behavior emerge unsupervised. The empty-think-block convention plus the "response adheres to the last flag" multi-turn rule (§4.3) is the mechanically simple but experimentally delicate trick: too little non-thinking data and the model refuses to stop thinking; too much and RL reasoning collapses. The report shows the balance point exists (Table 22) but discloses none of the mixture ratios that hit it.

关键实现细节 (1–2 easy-to-miss tricks).

1. Empty-think-block for non-thinking data (§4.3, Table 9): non-thinking samples still carry a \n\n block (empty), so the model always sees the same structural scaffold — this is what makes budget-based early-halt (inject mid-stream) coherent rather than out-of-distribution.
2. Entropy control during Reasoning RL (§4.2): the authors call out keeping model entropy "steadily increasing or stable" as crucial for stable training, combined with large batch + high rollouts-per-query + off-policy reuse for sample efficiency — a small note that governs whether the 170-step 70.1→85.1 jump reproduces.

Reproducibility & ecosystem. Training code closed; closest open references for the recipe are verl / OpenRLHF / TRL for GRPO-style RL, and standard KL-distillation loops for the on-policy stage. The data-mixture novelty (instance-level optimization via small proxy models, contrasted with DoReMi/DoGE/RegMix which work at source/domain level) is described but not released. Qwen3 weights are widely deployed and the /think–/no_think template is natively supported by the HF tokenizer, so the inference-time behavior is fully reproducible even though the training is not.