slmsql-grpo-v2 — GRPO on Text-to-SQL (a documented negative result)

TL;DR — this adapter does NOT beat its SFT starting point. It is published as an honest, reproducible negative result: outcome-only GRPO gave ~0 Execution-Accuracy gain on BIRD/Spider, even after we fixed the reward- variance problem that is the usual scapegoat. The value here is the finding and the clean telemetry that proves the mechanism worked but the accuracy did not move. For the actual best model, use Anurich/slmsql-qwen2.5-coder-1.5b.

Results

Why publish a null result?

1. Attempt #1 — GRPO on Spider-train. The model was already ~82% on Spider, so most K-sample groups were all-correct → frac_reward_zero_std ≈ 0.7 (70% of groups had zero reward variance → zero advantage → no gradient). Flat EX. This could be dismissed as "you fed it data it had already mastered." GRPO's advantage is (r − mean)/std; an all-same-reward group has std = 0 → no learning signal. This is the textbook GRPO failure mode.
2. Attempt #2 — GRPO on difficulty-filtered data (this run). We fixed the confound with a DAPO-style pass-rate filter: sampled the SFT model K=8× on 1,367 held-out SynSQL questions and kept only the uncertainty band (0.2 < pass_rate < 0.8). Result: 494 band / 502 all-fail / 184 all-pass — a healthy, balanced difficulty split. GRPO on the 494 band then trained with a perfectly healthy mechanism: frac_reward_zero_std fell to 0.05–0.2, reward_std ≈ 0.44, grad_norm ≈ 0.5, gradient flowing every step.
   And the EX still did not move (table above).

Training details

• Base policy: two-stage SFT LoRA adapter (SynSQL foundation → BIRD+Spider in-domain grounding), rank 32 / scale 64, all-linear target modules.
• RL data: 494 pass-rate-filtered SynSQL questions (0.2 < p < 0.8), execution-verifiable (ship runnable SQLite DBs).
• Algorithm: TRL GRPOTrainer, 8 generations/prompt, beta=0 (no reference model), lr=1e-6 (cosine decay), temperature=0.8, epsilon=0.2, use_vllm=False. 400 steps, ~2h42m on a single 24–32 GB NVIDIA card.
• Reward (Arctic-Text2SQL-R1 shape, deliberately simple): 1.0 if the SQL executes and the result matches gold; 0.1 if it executes but is wrong; 0.0 otherwise. Plus a tiny 0.2 format reward for a ```sql block. Sandbox: read-only connection + op-budget interrupt for adversarial SQL.

Intended use & limitations

• Use as a research artifact / reference for what outcome-only GRPO does (and does not do) on Text-to-SQL, and as a baseline for stronger RL (process-reward GRPO) or for distillation (the pivot we took after this result).
• Do not expect it to outperform the SFT model. It is functionally equivalent in accuracy and slightly terser in output length (GRPO nudged completions to ~80 tokens).
• Small-slice evaluation (149 BIRD / 200 Spider); differences under ~4% are noise.

Reproduce