xorazm-text2sql-0.8b (GRPO Checkpoint)
This directory contains the final checkpoint files for xorazm-text2sql-0.8b, a lightweight SQLite text-to-SQL translation model. The model was fine-tuned using Group Relative Policy Optimization (GRPO) reinforcement learning on a unified dataset.
Developer & Organization
Training Setup
The training pipeline was executed via Unsloth.
Hyperparameters:
- Base Model:
Qwen/Qwen3.5-0.8B
- Frameworks: Unsloth (for memory-efficient fast training) & TRL
GRPOTrainer
- Loss Type:
dr_grpo
- Optimizer:
adamw_8bit (8-bit AdamW)
- Learning Rate:
5e-6 with Cosine scheduler
- Warmup Ratio:
0.03
- Weight Decay:
0.1
- Max Prompt Length:
6,140 tokens (filtered from maximum 6,144 context window)
- Max Completion Length:
512 tokens
- Generations per Prompt (
num_generations): 8
- Gradient Accumulation Steps:
2
Dataset
- Training Dataset: BIRD; SPIDER; SYNSQL-2.5M
- Original Examples: 20,000 for SFT and 16,676 for GRPO (pruned down to 15,566 after filtering prompts exceeding 6,140 tokens using
tokenizer.apply_chat_template check)
- Data Sources: Unified blend of:
- BIRD Bench (Spider / Bird style SQL DDL + sample rows)
- Spider 1.0
- synSQL-2.5M
Reward Metrics & Weights
Three independent reward signals were utilized:
- Execution Accuracy Reward (
execution_reward_func): Binary check on whether the query successfully executes and matches the output rows of the gold query.
- Format Compliance Reward (
format_reward_func): Checks for proper </think>\n ```sql ... ``` tag adherence.
- Syntax Reward (
syntax_reward_func): Verifies SQL syntax correctness.
Compute & Training Statistics
- Hardware: 1x NVIDIA L40S GPU (44.394 GB Max VRAM)
- VRAM Before Training: 1.617 GB reserved
- Epochs: 1
