This is Version 2 of the sirunchained/text-to-sql-model, fine-tuned from google/gemma-3-270m-it for Text-to-SQL generation.
In this version, the model is merged with the LoRA adapter – you can load it directly with pipeline() (no PEFT required).

Key improvements in v2:

Feature	v1 (LoRA Adapter)	v2 (Merged)
Load method	Required PEFT + base model	Direct `pipeline()`
Model size	~10 MB (adapter only)	~536 MB (full model)
Inference speed	Slower (requires adapter load)	Faster
Ease of use	Complex	Simple
Performance	89.7% accuracy	✅ Same

🧠 Task

Text-to-SQL Generation
Converts natural language questions into SQL queries. Supports:

✅ SELECT queries (with JOINs, aggregations, subqueries)
✅ INSERT operations
✅ UPDATE operations
✅ DELETE operations (currently weak at this)

📊 Training Details

Item	Value
Base Model	`google/gemma-3-270m-it`
Fine-tuning Method	LoRA + 4-bit quantization (QLoRA)
Framework	`trl` (SFTTrainer)
Dataset	`sirunchained/text-to-sql-dataset` (4518 samples training, 200 validation, 200 test)
Training Epochs	5
Batch Size	32
Learning Rate	5e-5
LoRA Rank (r)	8
LoRA Alpha	16
Optimizer	AdamW (fused)

📈 Training Performance

Epoch	Training Loss	Validation Loss	Mean Token Accuracy
1	0.800	0.700	83.8%
2	0.650	0.680	84.2%
3	0.500	0.650	84.6%
4	0.350	0.640	85.1%
5	0.550	0.640	83.5%

Best validation loss was achieved at epoch 4 & 5 (0.640).
Highest mean token accuracy on validation was at epoch 4 (85.1%).

💻 Quick Start

Using Pipeline (Recommended)

python

from transformers import pipeline

generator = pipeline(
    "text-generation",
    model="sirunchained/text-to-sql-model-v2",
    device=0  # or "cuda"
)

# Example with schema
prompt = """<start_of_turn>user
# Schema
customers(id, name, email, country)
# Text
Find customers from USA.<end_of_turn>
<start_of_turn>model
"""
result = generator(prompt, max_new_tokens=128)
print(result[0]["generated_text"])

With Chat Template

python

from transformers import pipeline

pipe = pipeline("text-generation", model="sirunchained/text-to-sql-model-v2")

messages = [
    {"role": "user", "content": "# Schema\ncustomers(id, name, email)\n\n# Text\nFind customers with gmail emails."}
]

outputs = pipe(
    pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True),
    max_new_tokens=128
)
print(outputs[0]["generated_text"])

🎯 Dataset

The model was trained on sirunchained/text-to-sql-dataset:

Split	Size
Train	4,518 samples
Validation	200 samples
Test	200 samples

Dataset format:

text: Natural language question
schema: Optional database schema
query: Target SQL query

🧪 Evaluation Results

Test Set Performance (Epoch 5 model):

Metric	Value
Test Loss	0.638
Mean Token Accuracy	0.830
Entropy	0.636

📝 Version History

Version	Date	Description
v1	2026-07-22	LoRA adapter only (not directly loadable with pipeline)
v2	2026-07-23	Merged version – fully loadable with `pipeline()`

🛠️ Training Configuration

python

# LoRA Configuration
LoraConfig(
    r=8,
    lora_alpha=16,
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.CAUSAL_LM,
)

# Training Configuration
SFTConfig(
    num_train_epochs=5,
    per_device_train_batch_size=32,
    learning_rate=5e-5,
    lr_scheduler_type="constant",
    weight_decay=0.0,
    load_best_model_at_end=True,
    metric_for_best_model="mean_token_accuracy",
    greater_is_better=True,
)

⚠️ Important Notes

This is a small language model (270M parameters) – works on T4 GPUs
Provide schema only when needed – works with or without it
For non-SQL requests, the model outputs INVALID_QUERY (trained with negative samples)
The model handles INSERT, UPDATE, and DELETE queries correctly

🔗 Links

Base Model: google/gemma-3-270m-it
Dataset: sirunchained/text-to-sql-dataset
v1 (Adapter): sirunchained/text-to-sql-model

🙏 Acknowledgments

Built with:

Hugging Face Transformers
TRL (Transformer Reinforcement Learning)
PEFT (Parameter-Efficient Fine-Tuning)
bitsandbytes
Gradio for the demo interface which you can use here

📄 License

This model is released under the same license as Google's Gemma model. See the Gemma model card for details.