You can deploy this model and start earning money today!

Ahmed-Ismail/
Qwen-1.5B-Text-to-SQL-LoRA

Name: Qwen-1.5B-Text-to-SQL-LoRA
Availability: InStock

transformers

safetensors

endpoints_compatible

Views

No views yet

Qwen 1.5B - Text-to-SQL LoRA

This is a fine-tuned LoRA adapter for Qwen/Qwen2.5-Coder-1.5B designed to convert natural language questions into strict, executable SQL queries based on a provided database schema.

💻 Project Goal

This model was trained as a portfolio project to demonstrate efficient fine-tuning of Large Language Models for structured data generation using PyTorch, PEFT, and 8-bit quantization (QLoRA) on constrained hardware.

🛠️ Training Details

Base Model: Qwen/Qwen2.5-Coder-1.5B
Dataset: Downsampled b-mc2/sql-create-context (25,000 highly curated schema/question/query pairs)
Technique: LoRA (Rank 16, Alpha 32) targeting all linear layers for maximum logical reasoning capability.
Infrastructure: Fine-tuned on a single NVIDIA T4 GPU using gradient checkpointing and an 8-bit Paged AdamW optimizer.

🚀 How to Use

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "Qwen/Qwen2.5-Coder-1.5B"
adapter_id = "your-username/Qwen-1.5B-Text-to-SQL-LoRA"

# Load the base model and tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(base_model_id)

# Apply the LoRA adapter
model = PeftModel.from_pretrained(base_model, adapter_id)

# Format your prompt using the strict schema template
prompt = """Schema:
CREATE TABLE employees (id VARCHAR, name VARCHAR, department VARCHAR, salary INTEGER)

Question:
How many employees in the engineering department make more than 100000?

SQL:
"""
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=150, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Ahmed-Ismail/Qwen-1.5B-Text-to-SQL-LoRA

Qwen 1.5B - Text-to-SQL LoRA

💻 Project Goal

🛠️ Training Details

🚀 How to Use

Ahmed-Ismail/
Qwen-1.5B-Text-to-SQL-LoRA