GGUF quantizations of jk200201/qwen2.5-coder-7b-bird-cot, a 7B text-to-SQL model that reasons step by step over a database schema and then writes the SQL. Runs locally on CPU or a single consumer GPU with llama.cpp or Ollama.
On BIRD dev (realistic, messy schemas, the harder text-to-SQL benchmark) the full-precision model reaches 52.1% greedy result accuracy and 58.5% with self-consistency (Best-of-N, K=8). With self-consistency it matches DeepSeek V4-Pro (1.6T parameters) at roughly 0.4% of the size, running locally.
Files
File
Quant
Size
Notes
qwen2.5-coder-7b-bird-cot-Q4_K_M.gguf
Q4_K_M
4.4 GB
Recommended. Best size/quality trade-off.
qwen2.5-coder-7b-bird-cot-Q5_K_M.gguf
Q5_K_M
5.1 GB
Higher fidelity, slightly larger.
qwen2.5-coder-7b-bird-cot-Q8_0.gguf
Q8_0
7.6 GB
Near lossless.
qwen2.5-coder-7b-bird-cot-f16.gguf
F16
15 GB
Full precision reference.
Start with Q4_K_M. Move up to Q5_K_M or Q8_0 if you have the memory and want to close the small accuracy gap.
Quick start (Ollama)
bash
1# Download the Q4_K_M file, then from the same directory:2ollama create bird-cot -f Modelfile
3ollama run bird-cot
Modelfile:
dockerfile
1FROM ./qwen2.5-coder-7b-bird-cot-Q4_K_M.gguf23TEMPLATE """{{ if .System }}<|im_start|>system
4{{ .System }}<|im_end|>
5{{ end }}{{ if .Prompt }}<|im_start|>user
6{{ .Prompt }}<|im_end|>
7{{ end }}<|im_start|>assistant
8{{ .Response }}<|im_end|>
9"""
1011SYSTEM """You are an expert SQLite query writer. Reason step by step about the schema and the question, then output the final query in a fenced sql code block."""
1213PARAMETER temperature 0
14PARAMETER num_ctx 8192
15PARAMETER stop "<|im_start|>"
16PARAMETER stop "<|im_end|>"
Then send a schema and a question:
Given the database schema and question, work out the correct SQLite query step by step.
Database Schema:
CREATE TABLE singer (Singer_ID INT, Name TEXT, Age INT);
Question: How many singers are older than 40?
Think step by step, then give the final answer in a fenced sql block.
The model returns its reasoning followed by a fenced SQL block. Take the last SQL block as the query.
Quick start (llama.cpp)
bash
1./llama-cli -m qwen2.5-coder-7b-bird-cot-Q4_K_M.gguf -no-cnv --temp 0 -n 512\2 -p $'<|im_start|>system\nYou are an expert SQLite query writer. Reason step by step about the schema and the question, then output the final query in a fenced sql code block.<|im_end|>\n<|im_start|>user\nGiven the database schema and question, work out the correct SQLite query step by step.\n\nDatabase Schema:\nCREATE TABLE singer (Singer_ID INT, Name TEXT, Age INT);\n\nQuestion: How many singers are older than 40?\n\nThink step by step, then give the final answer in a fenced sql block.<|im_end|>\n<|im_start|>assistant\n'
Results: BIRD dev (execution result accuracy)
All models use a single chain-of-thought sample unless noted, so the comparison is like for like.
Model
Params
Result accuracy
Base Qwen2.5-Coder-7B-Instruct
7B
27.0%
This model, greedy
7B
52.1%
This model, self-consistency (K=8)
7B
58.5%
DeepSeek V4-Pro
1.6T
58.7%
GLM 5.2
744B
63.0%
Result accuracy is the fraction of queries whose SQL executes to the same rows as the gold query (BIRD's official execution metric). The reported figures are for the full-precision model. Quantization retains most of that accuracy; Q4_K_M is the recommended balance.
Training
Reasoning distillation (CoT-SFT). A strong teacher (Qwen3-Coder-480B) generated step-by-step CoT solutions on BIRD train. Only execution-verified-correct chains were kept (5,593 examples), then supervised fine-tuned into the 7B. Distilling the teacher's reasoning generalized across BIRD's cross-domain dev databases better than distilling SQL answers directly.
