qwen3.5-9b-text2sql-grpo
An agentic text-to-SQL model: given a natural-language question and a
SQLite database, it explores the schema with tools, runs candidate queries,
observes real execution feedback, and submits a final SQL query.
Trained with
agentic GRPO directly from vanilla Qwen/Qwen3.5-9B — no
supervised finetuning stage, no teacher traces. 200 GRPO steps in an
execution-grounded environment (
verifiers
/
prime-rl): the reward
executes the submitted query against the real database and compares result
sets to gold (1.0 match / 0.1 runs-but-wrong / 0.0 error / −1.0 write
attempt). Training prompts come from the BIRD train split, filtered to those
the base policy solves 1–7 times out of 8 (prompts with no gradient signal
are dropped).
Results (BIRD dev, 1,534 examples, execution accuracy)
| setting | EX |
|---|
| agentic, greedy | 63.4% |
| agentic, exec-consistency voting @ k=8 / k=32 (temp 1.0) | 67.0% / 67.4% |
| single-turn (no tools), greedy | 58.2% |
base Qwen/Qwen3.5-9B zero-shot agentic, for reference | 49.7% |
Exec-consistency voting: sample k rollouts, group the submitted queries by
the result set they return, exclude empty results, take the largest group
(majority over result sets — never over SQL text, and without de-duplicating
verbatim repeats).
Usage
The model calls tools in Qwen3.5's XML dialect — serve with vLLM using
--enable-auto-tool-choice --tool-call-parser qwen3_xml (the hermes
parser silently parses nothing).
It expects the agentic environment's tool set:
inspect_schema,
sample_rows,
run_sql (read-only),
submit. The training/eval harness,
environment definition, and a ready-to-use CLI (
scripts/ask.py, agentic
and single-turn modes, works against any SQLite file) live in the training
repository:
https://github.com/oliverkinch/text2sql-envIt also works as a plain single-turn SQL generator (schema in the prompt,
SQL out) at the lower accuracy listed above.
Limitations
- Tuned and measured on SQLite only.
- Rewarded only for read queries; it was penalized for writes, but do not
point it at a database you cannot afford to mutate without a read-only
connection (the training harness enforced one).
- BIRD dev was used for model selection across the project's history;
treat dev numbers as development metrics, not test claims.
