DeepScaleR → 16-token soft prompt (negative result)
A 16-token soft prompt distilled from the frozen published RL checkpoint
agentica-org/DeepScaleR-1.5B-Preview onto the frozen published base
deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B.
No RL was run. The RL step was completed upstream by the Agentica team; this artifact
only samples the frozen RL model and trains prompt embeddings on the frozen base.
This prompt does not recover the RL gain. It is published because the negative result,
together with the measurements below, is the informative part.
Artifact
| |
|---|
| tensor | prompt_embeddings, BF16, shape [16, 1536] |
| sha256 | 84258d729ea6a876bec95b1a7978b6e9e79fec3e65263c4f7b95ad239fdbef36 |
| training | 100 steps (2 epochs), effective batch 512, lr 1e-2 constant, Gaussian init std 0.02 seed 12345 |
| data | 25,549 correct-only trajectories (cap 8/question) from 3,481 of 4,096 questions |
Result
Pooled AIME 2024+2025+2026 (90 questions, n=16, temperature 0.6, top_p 0.95, 32K,
rm_hub.deepscaler grader, rllm system prompt):
| condition | pass@1 |
|---|
| base | 24.17% |
| DeepScaleR (RL) | 32.36% |
| base + this prompt | 24.51% |
recovery@1 = 3.12%, 95% CI [-25.53%, +24.79%] — indistinguishable from zero.
Why it fails, measured
The training objective was almost fully solved. Measured on 400 held-out teacher
trajectories:
| NLL | mean P(next token) |
|---|
| base | 0.27262 | 76.14% |
| base + prompt | 0.22456 | 79.89% |
| DeepScaleR (floor) | 0.22213 | 80.08% |
The prompt closed 95.2% of the achievable gap. So 16 tokens are not a capacity
limitation — the objective itself is the problem. Base and teacher differ by only ~4
percentage points of next-token probability on the teacher's own correct trajectories.
The RL gain lives in which trajectories get sampled, not in how a correct one is worded, and
correct-only token-level imitation cannot express that.
The clearest symptom is length. RL made responses shorter (46,184 → 28,316 characters,
truncation 17.8% → 2.4%); the prompt made them longer (58,462 characters, truncation
29.3%). RL's brevity comes from suppressing rambling incorrect trajectories — information
that a correct-only snapshot has thrown away.
Contents
prompt.safetensors — the step-100 promptmetrics.jsonl — per-step loss, gradient norm, token countnll_floor.json — the floor measurement abovemath_training_curve.png — loss against the measured floor
