You can deploy this model and start earning money today!

revanthkumar1999/
Llama3_de

Name: Llama3_de
Availability: InStock

peft

safetensors

llama

meta-llama/Llama-3.1-8B

adapter

Views

No views yet

Model Card for LLaMA 3.1 8B - QLoRA Fine-Tuned for Text-to-SQL

Model Details

Model Description

This model is a fine-tuned version of meta-llama/Llama-3.1-8B optimized for text-to-SQL generation using the QLoRA method.
Fine-tuning was performed on a curated dataset of natural language database queries and their corresponding SQL statements.
Training was conducted on NVIDIA RTX 4090 GPU hardware, with compute resources and technical guidance provided by San Jose State University.

Developed by: Revanth Kumar Bondada
Institutional Support: San Jose State University
Model type: Causal Language Model (decoder-only Transformer)
Language(s): English
License: Same as base model (meta-llama/Llama-3.1-8B)
Finetuned from: meta-llama/Llama-3.1-8B

Model Sources

Repository: GitHub Link
Hugging Face Model: Model Link

Uses

Direct Use

Convert natural language database queries to SQL statements.
Educational and research purposes in NLP-to-database mapping.

Downstream Use

Integrating into applications that require NL-to-SQL translation.
Fine-tuning further for domain-specific databases.

Out-of-Scope Use

Generating SQL for malicious purposes (SQL injection attacks).
Queries in non-English languages (not trained/tested for these).

Bias, Risks, and Limitations

Performance may drop for highly domain-specific schemas unseen during training.
Possible generation of incorrect SQL syntax or queries that don't match user intent.

Recommendations

Always validate generated SQL before execution in production.
Consider schema-aware fine-tuning for production-grade performance.

How to Get Started with the Model

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "revanthkumar1999/Llama3_de"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")

prompt = "List all customers who placed an order in the last 30 days."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

Curated dataset combining open-source text-to-SQL corpora and custom synthetic data.

Training Procedure

Method: QLoRA
Precision: 4-bit quantization during training
Optimizer: AdamW
Learning rate: 2e-4
Batch size: 16
Epochs: 3
Context length: 4096 tokens

Compute Infrastructure

Hardware: NVIDIA RTX 4090 GPU (24GB VRAM)
Framework: PyTorch + PEFT + Transformers
Environment: Local workstation with CUDA 12.1

Evaluation

Testing Data

Split from the training set plus additional unseen SQL tasks.

Metrics

Exact Match (EM)
Execution Accuracy (EX)
BLEU Score

Results

Metric	Score
Exact Match	83%
Execution Acc.	88%
BLEU	91

Environmental Impact

Hardware Type: NVIDIA RTX 4090
Training Time: ~8 hours
Location: San Jose, CA
Carbon Emitted: ~4.2 kg CO₂eq (estimated)

Model Architecture

8B parameter LLaMA 3.1 decoder-only Transformer
QLoRA adapters for efficient fine-tuning

Framework Versions

Transformers 4.43.0
PEFT 0.15.2.dev0
PyTorch 2.3.0