It uses PEFT (LoRA) to adapt the base model efficiently with minimal extra parameters.
Useful for learning and prototyping text-to-SQL tasks on simple table schemas.

📚 Training Details

Base Model: Salesforce/codet5-small
Adapter: LoRA (r=8, alpha=16) on attention q and v modules.
Dataset: WikiSQL (21k train, 3k val)
Input Format: question: <QUESTION> table: <TABLE_HEADERS>
Target: Human-readable SQL query
Epochs: 1–3 recommended for small runs.
Framework: 🤗 Transformers + PEFT

🧩 Example Usage

python

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

# Replace with your actual HF repo name
model_name = "Mahendra1742/SqlGPT"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

# Example input
question = "How many employees are in the Marketing department?"
table = "| department | employees |"

prompt = f"question: {question} table: {table}"

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(**inputs, max_length=128)

print("OUTPUT :- ")
print("   ")
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Input

question: How many cities have a population over 1 million? table: | City | Population |

Output

SELECT COUNT(*) FROM table WHERE Population > 1000000