The model was fine-tuned using QLoRA with a 4-bit quantized base model and LoRA adapters. After training, the adapter was merged with the base model to produce this standalone model.

Model Repositories

Merged Model

The base model and trained LoRA adapter have been merged into a standalone model:

https://huggingface.co/farehaheha/llama3.2-3B-text-to-sql

Quantized GGUF Model

A Q4_K_M GGUF quantized version is available for efficient local inference with llama.cpp and other GGUF-compatible runtimes:

https://huggingface.co/farehaheha/llama3.2-3B-text-to-sql-Q4_K_M-GGUF

Training Details

Parameter	Value
Base Model	`meta-llama/Llama-3.2-3B`
Fine-tuning Method	QLoRA
Base Model Quantization During Training	4-bit
LoRA Rank	16
Training Dataset	`gretelai/synthetic_text_to_sql`
Dataset Split Used	Test split
Training Samples	~5.85K
Epochs	1
Batch Size	8
Training Time	~30 minutes

Note: This experiment used the dataset's test split for fine-tuning rather than the training split. Therefore, the original test split should not be used to report an unbiased evaluation score for this model.

Dataset

The model was fine-tuned on:

Gretel AI Synthetic Text-to-SQL

https://huggingface.co/datasets/gretelai/synthetic_text_to_sql

The dataset contains natural-language questions paired with SQL queries and database context.

Approximately 5,850 examples from the test split were used for this fine-tuning experiment.

Fine-Tuning Approach

The training process used QLoRA:

The Llama 3.2 3B base model was loaded in 4-bit precision.
The original base model weights remained frozen.
LoRA adapters with rank 16 were trained on the Text-to-SQL dataset.
Training was performed for one epoch.
The trained LoRA adapter was merged with the base model.
The merged model was also converted to Q4_K_M GGUF for local inference.

This approach significantly reduces the memory required for fine-tuning compared with full-parameter training.

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "farehaheha/llama3.2-3B-text-to-sql"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype="auto",
)

prompt = """### Database Schema:
{your_database_schema}

### Request:
{your_natural_language_request}

### SQL Query:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    do_sample=False,
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)