A LoRA fine-tune of Qwen/Qwen2.5-1.5B-Instruct for text-to-SQL generation: given a database schema and a natural-language question, the model generates the corresponding SQL query. Weights in this repo are the base model with the trained LoRA adapter merged in, so it loads and runs like any standard causal LM — no PEFT wrapper required.

Model description

Base model: Qwen2.5-1.5B-Instruct
Fine-tuning method: QLoRA (4-bit base, LoRA rank 16, alpha 32, targeting q_proj/k_proj/v_proj/o_proj)
Task: Text-to-SQL generation
Training hardware: Single T4 GPU (Google Colab free tier)
Training framework: 🤗 Transformers, PEFT, TRL (SFTTrainer)

Training data

b-mc2/sql-create-context — natural-language question, database schema (CREATE TABLE statement), and gold SQL query triples, derived from WikiSQL and Spider. Trained on a 20,000-example subset (randomly sampled, seed 42) for time budget on free-tier compute; a 95/5 train/test split was used.

Training procedure

1 epoch, effective batch size 16 (per-device batch 4 × gradient accumulation 4)
Learning rate 2e-4
Max sequence length 512 tokens
bfloat16 compute dtype, 4-bit NF4 base quantization with double quantization
Prompt format: Qwen's ChatML template, with a system prompt instructing SQL-only output (no explanation)

Evaluation

Evaluated on a 300-example held-out sample from the test split, using two metrics:

Metric	Score
Exact-match accuracy	`73.0%`
Execution validity rate	`96.7%`
Gold query validity (sanity ceiling)	`97.7%`

Exact-match accuracy: generated SQL equals gold SQL after normalizing whitespace/case.

Execution validity rate: percentage of generated queries that execute without error against an in-memory SQLite database built from the example's real schema. Note: b-mc2/sql-create-context provides schemas but no populated rows, so this measures syntactic/semantic validity against the real schema (correct table/column references, valid SQL) rather than whether results are correct — not full execution accuracy. The gold queries' own validity rate is reported as a sanity ceiling.

How to use

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "yuashi/qwen2.5-1.5b-text2sql-merged"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

schema = "CREATE TABLE employees (id INT, name TEXT, department TEXT, salary INT)"
question = "What is the average salary in the engineering department?"

messages = [
    {"role": "system", "content": "You are a SQL expert. Given a database schema and a question, write the correct SQL query. Respond with only the SQL query, no explanation."},
    {"role": "user", "content": f"Schema:\n{schema}\n\nQuestion: {question}"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=150, temperature=0.1, do_sample=False)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

A quantized GGUF version (Q4_K_M, ~1GB, CPU-inference friendly via llama.cpp) is also available at: https://huggingface.co/yuashi/qwen2.5-1.5b-text2sql-gguf.

Limitations

Trained on a 20k subset of the available ~78k examples
Execution validity is a proxy metric, not true execution accuracy (no populated rows in the training/eval data to check result correctness against)
No adversarial or out-of-distribution schema testing performed
Inherits any limitations and biases of the base Qwen2.5-1.5B-Instruct model

License

Apache 2.0, inherited from the base model. Training dataset (b-mc2/sql-create-context) is CC-BY-4.0.