This model was fine-tuned on 2000 synthetic Text-to-SQL examples generated using a teacher model (DeepSeek V3). The fine-tuning was performed using LoRA adapters with MLX on Apple Silicon, then fused into the base model.

Training Details

Base Model: LiquidAI/LFM2.5-1.2B-Instruct
Training Data: 2000 synthetic examples
Training Method: LoRA fine-tuning (FP16)
Iterations: 5400
Hardware: Apple Silicon (MLX)

Performance

Model Comparison

Metric	Teacher (DeepSeek V3)	Base Model	Fine-tuned
Exact Match	60%	48%	72%
LLM-as-Judge	90%	75%	87%
ROUGE-L	92%	83%	94%
BLEU	85%	70%	89%
Semantic Similarity	96%	93%	97%

Training Progression

The model shows consistent improvement across all checkpoints with no signs of overfitting.

Usage

PyTorch / Transformers

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "hybridaione/LFM2.5-1.2B-Text2SQL",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("furukama/LFM2.5-1.2B-Text2SQL", trust_remote_code=True)

# Example query
prompt = '''CREATE TABLE employees (id INT, name VARCHAR, salary DECIMAL);

Question: What are the names of employees earning more than 50000?'''

messages = [{"role": "user", "content": prompt}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

License

This model is released under the Apache 2.0 license, following the base model's license.