Views
No views yet
1{
2 "base_model": "google/gemma-7b",
3 "method": "LoRA",
4 "rank": 16,
5 "alpha": 32,
6 "dropout": 0.05,
7 "target_modules": ["q_proj", "k_proj", "v_proj", "o_proj"],
8 "epochs": 3,
9 "batch_size": 8,
10 "learning_rate": 1.5e-4,
11 "training_time": "10.8 hours (A100 GPU)"
12}Epoch 1: Loss 1.456 | Val Loss 1.512 | Accuracy 68.2%
Epoch 2: Loss 0.521 | Val Loss 0.589 | Accuracy 72.8%
Epoch 3: Loss 0.234 | Val Loss 0.267 | Accuracy 76.0%pip install transformers torch1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model = AutoModelForCausalLM.from_pretrained("estu-research/gemma-7b-sql-ft")
4tokenizer = AutoTokenizer.from_pretrained("estu-research/gemma-7b-sql-ft")
5
6# Example query
7question = """
8Schema: CREATE TABLE customers (customerNumber INT, customerName VARCHAR(50), country VARCHAR(50));
9Question: List all customers from France
10"""
11
12inputs = tokenizer(question, return_tensors="pt")
13outputs = model.generate(**inputs, max_new_tokens=256)
14sql = tokenizer.decode(outputs[0], skip_special_tokens=True)
15
16print(sql)
17# Output: SELECT * FROM customers WHERE country = 'France';1from transformers import pipeline
2
3pipe = pipeline("text-generation", model="estu-research/gemma-7b-sql-ft")
4
5result = pipe(
6 "Schema: CREATE TABLE products (productName VARCHAR, price DECIMAL);\nQuestion: Show top 10 expensive products",
7 max_new_tokens=200,
8 temperature=0.1
9)
10print(result[0]['generated_text'])| Natural Language | Generated SQL |
|---|---|
| List top 5 customers by sales | SELECT customerName, SUM(amount) as total FROM customers JOIN orders USING(customerId) GROUP BY customerId ORDER BY total DESC LIMIT 5; |
| Show products never ordered | SELECT p.productName FROM products p LEFT JOIN orderDetails od ON p.productCode = od.productCode WHERE od.productCode IS NULL; |
| Total revenue by country | SELECT country, SUM(amount) as revenue FROM customers JOIN orders USING(customerId) GROUP BY country ORDER BY revenue DESC; |
| Model | Accuracy | Latency | Cost |
|---|---|---|---|
| Gemma-7B (FT) | 76.0% | 500ms | Free |
| Llama-3-8B (FT) | 78.2% | 450ms | Free |
| GPT-4o-mini (FT) | 97.8% | 800ms | $0.30/1K |
| GPT-3.5 Turbo | 78.9% | 500ms | $0.05/1K |
1@misc{gemma7b-sql-ft,
2 title={Gemma-7B SQL Expert: Fine-Tuned Model for Text-to-SQL},
3 author={Kulalı and Aydın and Alhan and Fidan},
4 institution={Eskisehir Technical University},
5 year={2024},
6 url={https://huggingface.co/estu-research/gemma-7b-sql-ft}
7}