Slonik-7B-SFT

A PostgreSQL-focused text-to-SQL model, fine-tuned from Qwen2.5-Coder-7B-Instruct via QLoRA on a mix of BIRD, Spider, Gretel, and 2,000 PostgreSQL-specific examples I synthesized for this run.

This is the SFT-only baseline. For better numbers, see:

Phani-labs/Slonik-7B-GRPO — same SFT base, then 2000 GRPO steps with execution rewards. 38.2% BIRD-PG, 45.2% BIRD-SQLite.
Phani-labs/Slonik-7B-GRPO-GGUF — GGUF quantizations (Q4_K_M, Q5_K_M, Q8_0) of the GRPO model for Ollama / llama.cpp / LM Studio.

Why I built this

I tested several text-to-SQL models on PostgreSQL-heavy schemas and most handled simple SELECT queries well but struggled once schemas involved JSONB fields, pgvector similarity, full-text search, array operators, or deeper CTE / window-function patterns. This is the SFT pass focused on those cases.

Results

BIRD Mini-Dev PostgreSQL (500-example official benchmark, execution accuracy):

Model	BIRD-PG	Size
GPT-4o	34.44%	proprietary
Slonik-7B-SFT (this)	33.20%	7B
Qwen2.5-Coder-32B	22.96%	32B
Codestral 22B	21.11%	22B
Qwen2.5-Coder-7B (base)	12.22%	7B

A 7B model approaching GPT-4o performance with pure SFT — and 10+ points above Qwen-32B on the same benchmark.

Difficulty breakdown

Simple: 48.6%
Moderate: 29.6%
Challenging: 19.6%

What it covers

The SFT data emphasizes PostgreSQL patterns that are often underrepresented in generic text-to-SQL datasets:

pgvector similarity operators such as <->, <=>, and <#>
JSONB access and containment patterns using ->, ->>, @>, and jsonb_path_query
full-text search with tsvector, @@, websearch_to_tsquery, and ranking functions
array operators such as &&, ANY(...), and array_length
CTEs, lateral joins, CASE expressions, COALESCE, and window functions

Training


Base model	Qwen2.5-Coder-7B-Instruct
Method	QLoRA SFT (4-bit base, LoRA adapters merged)
LoRA rank / alpha	32 / 64
Target modules	`q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj`
Trainable params	80.7M (1.05% of base)
Max seq length	1024
Optimizer	`adamw_torch_fused`
Learning rate	1e-5, cosine schedule, 10% warmup
Max grad norm	0.5
Epochs	3
Effective batch	16 (per-device 2 × grad_accum 8)
Precision	bf16
Hardware	RTX 5080 Laptop 16 GB (Blackwell sm_120)
Train time	8h 13min

Training data (21,847 examples)

BIRD-SQL train split — 6,601 examples
Spider train split — 8,034 examples
Gretel synthetic text-to-SQL PostgreSQL subset — 5,212 examples (filtered)
PG-Modern custom synthesis — 2,000 examples covering pgvector, JSONB, full-text search, CTEs, window functions, and array operations. Generated via DeepSeek-V4 with sqlglot-based PostgreSQL parseability validation.

Loss curve

Epoch	eval_loss
0.19	0.405
0.56	0.327
1.12	0.302
1.49	0.299
2.05	0.292
2.60	0.290
2.98	0.2896

Eval loss flattened around 0.29 near the end of training. I did not see obvious overfitting in this run; the train/eval loss gap stayed under 0.10, with final train loss at 0.226.

How to use

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "Phani-labs/Slonik-7B-SFT",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("Phani-labs/Slonik-7B-SFT")

schema = """
CREATE TABLE orders (id INT, user_id INT, total NUMERIC, created_at TIMESTAMPTZ);
CREATE TABLE users (id INT, name TEXT, country TEXT);
"""
question = "For each country, total revenue in 2025, ranked. Use a CTE."

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": f"Schema:\n{schema}\n\nQuestion: {question}"}],
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=400, do_sample=False, temperature=0.0)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Example output:

sql

WITH revenue_by_country AS (
    SELECT u.country, SUM(o.total) AS total_revenue
    FROM orders o
    JOIN users u ON o.user_id = u.id
    WHERE EXTRACT(YEAR FROM o.created_at) = 2025
    GROUP BY u.country
)
SELECT country, total_revenue,
       RANK() OVER (ORDER BY total_revenue DESC) AS revenue_rank
FROM revenue_by_country;

Limitations

A few limitations I noticed while manually reviewing harder PG-Modern evaluation examples:

Drops longer-tail constraints. On questions with four or five requirements, the model often gets the overall structure right but misses one filter (for example, forgets AND p.id != ref_id, or skips a IS NOT NULL check the gold SQL includes).
Prefers flat SELECTs over deep CTE chains when both are valid. The gold SQL may use three CTEs; Slonik often collapses to one CTE plus subqueries that compute the same thing.
Inconsistent function variants. May reach for to_tsquery when websearch_to_tsquery would match the intent more closely (or vice versa). Both parse, but semantics differ for multi-word queries.
Schema-dependent JSON paths. When a JSON field name is not directly given in the schema, the model sometimes guesses (for example, using a 'feature_count' key instead of computing count(*) over jsonb_path_query).

The GRPO variant (Phani-labs/Slonik-7B-GRPO) addresses some of these — particularly the schema-grounding and dialect issues — through execution-based reinforcement learning.

Notes for Blackwell laptops

On my RTX 5080 laptop the vLLM CUDA 13 wheels did not work cleanly with the CUDA 12.x runtime, so I trained through Unsloth's Triton fallback (no flash-attn, no nvcc). AdamW 8-bit produced NaNs within the first 100 steps every time; adamw_torch_fused with LR 1e-5 and grad clipping at 0.5 was what stabilized the run.

Author

Phani

GitHub: slonik-7b
GRPO variant (better accuracy): Phani-labs/Slonik-7B-GRPO
GGUF quantizations: Phani-labs/Slonik-7B-GRPO-GGUF