LoRA adapter fine-tuning IBM Granite 4.0 350M for text-to-SQL generation: given a CREATE TABLE schema and a natural-language question, generate the corresponding SQL query.

Model Details

Model Description

This checkpoint (step 6000 of 6432, epoch 2.80/3) is the best-performing checkpoint of the run — lowest eval loss among all saved checkpoints (see Results).

Model type: LoRA adapter (PEFT) on a causal LM
Language(s): SQL (generation), English (instructions)
License: Apache 2.0 (inherited from base model)
Finetuned from model: unsloth/granite-4.0-350m-unsloth-bnb-4bit (IBM Granite 4.0, 350M, GraniteMoeHybrid)

How to Get Started with the Model

python

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="/workspace/outputs/granite_sql_train/checkpoint-6000",
    load_in_4bit=False,
    device_map="cuda:0",
)
FastLanguageModel.for_inference(model)

instruction = """Generate ONLY the SQL query for the following database.

Do not explain your answer.
Do not include markdown.
Do not include any additional text.

Schema:
{schema}

Question:
{question}

SQL:"""

messages = [{"role": "user", "content": [{"type": "text", "text": instruction.format(schema=schema, question=question)}]}]
input_text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = tokenizer(input_text, add_special_tokens=False, return_tensors="pt").to("cuda")
output = model.generate(**inputs, max_new_tokens=256, use_cache=True, temperature=0.7, top_p=0.8, top_k=20)

Training Details

Training Data

b-mc2/sql-create-context — schema + natural-language-question + SQL-answer triples. Split via train_test_split(test_size=10000, seed=42, shuffle=True): remaining rows for train, 10,000 held out for eval.

Training Procedure

LoRA (r=16, alpha=32, dropout=0, bias=none, no rslora) applied to q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, shared_mlp.input_linear, shared_mlp.output_linear, trained with Unsloth + TRL SFTTrainer.

Training Hyperparameters

Training regime: bf16 mixed precision
Epochs: 3 | Per-device train batch size: 32 | Per-device eval batch size: 64 | Grad accumulation: 1
Learning rate: 2e-4, linear schedule, 50 warmup steps
Optimizer: adamw_8bit, weight decay 0.001, max grad norm 1.0
Seed: 3407 | Eval every 200 steps | Save every 500 steps

Speeds, Sizes, Times

Full run: 6432 steps / 3 epochs, train_runtime ≈ 2567s. This adapter checkpoint: ~26.7MB (adapter_model.safetensors).

Evaluation

Testing Data & Metrics

10,000-row held-out split of b-mc2/sql-create-context (see Training Data), evaluated by SFT eval loss (cross-entropy) every 200 steps.

Results

Checkpoint	Step	Eval loss
checkpoint-4000	4000	0.02830
checkpoint-5000	5000	0.02811
checkpoint-6000	6000	0.02673 (best saved)
checkpoint-6432 (final)	6432	~0.02746 (nearest eval at step 6400)

Lowest eval_loss observed during training was 0.02630 at step 5200, but no checkpoint was saved at that exact step (checkpoints every 500 steps, eval every 200), so checkpoint-6000 is the closest usable minimum. load_best_model_at_end was not enabled, so this checkpoint was selected manually by comparing eval_loss across saved checkpoints.

Compute Infrastructure

Hardware

1× NVIDIA A100 40GB

Software

Unsloth
Transformers
PyTorch

Model Card Authors

Yian

Contact

https://github.com/Siruenyian

siruenyian/sqlgenerator-mini

Model Card for granite_sql (checkpoint-6000)