SS-350M-SQL-Strict is a specialized, lightweight LLM fine-tuned for the singular task of Text-to-SQL translation. Built upon the LiquidAI LFM2.5-350M architecture, this model has been engineered to follow a "Strict" output protocol: it generates only raw SQL code, eliminating the conversational filler, Markdown blocks, and explanations typically found in general-purpose models.
By leveraging 4-bit QLoRA and Unsloth optimizations, this model provides high-speed, low-latency SQL generation suitable for edge deployment and resource-constrained environments.
Model Details
Developed by: Saad Salman
Architecture: Liquid Foundation Model (LFM) 2.5
Parameters: 350 Million
Quantization: 4-bit (bitsandbytes)
Fine-tuning Method: QLoRA
Primary Task: Natural Language to SQL (Strict)
Training Logic & Parameters
The model was trained using a custom pipeline to enforce strict code generation. The key differentiator is the use of Completion-Only Loss masking, which prevents the model from wasting weights on learning the prompt structure, focusing 100% of its learning capacity on the SQL syntax.
Hyperparameters
Parameter
Value
Description
Max Steps
800
Optimal convergence point for 350M params
Learning Rate
2e-4
High enough for rapid logic acquisition
Batch Size
16
(4 per device with 4 grad accumulation)
Rank (r)
32
High rank to capture complex SQL logic
Alpha
32
Scaling factor for LoRA weights
Optimizer
AdamW 8-bit
Memory-efficient optimization
Training Curve Analysis
The model demonstrated a classic "L-shaped" convergence curve. Initial loss started at ~38.1 and successfully plateaued between 8.0 and 11.0. This plateau indicates the model has fully internalized the ChatML structure and the SQL schema-mapping logic.
Prompting Specification (ChatML)
To ensure the "Strict" behavior, you must use the following ChatML format. Failure to use this format may result in hallucinated text.
Template
text
1<|im_start|>system
2You are a SQL translation engine. Return ONLY raw SQL. Schema: {YOUR_SCHEMA}<|im_end|>
3<|im_start|>user
4{YOUR_QUESTION}<|im_end|>
5<|im_start|>assistant
Example Input
text
1<|im_start|>system
2You are a SQL translation engine. Return ONLY raw SQL. Schema: Table 'orders' (id, price, status, created_at)<|im_end|>
3<|im_start|>user
4Find the average price of all 'completed' orders.<|im_end|>
5<|im_start|>assistant
Example Output
SELECT AVG(price) FROM orders WHERE status = 'completed';
Training Dataset
The model was trained on the Gretel Synthetic SQL dataset. This dataset is designed to cover:
Complex joins and subqueries.
Diverse industry domains (Finance, Retail, Tech).
Correct handling of GROUP BY, ORDER BY, and HAVING clauses.
Technical Limitations
Schema Size: Best suited for schemas with < 20 tables.
Dialect: Defaulted to standard SQL.
Reasoning: The model does not "explain" its code; it is a direct translation engine.
