Views
No views yet
July 31, 2025: Upload model to modelscope and huggingface.July 30, 2025: Publish the paper to arxivLarge language models (LLMs) have demonstrated strong performance in translating natural language questions into SQL queries (Text-to-SQL). In contrast, small language models (SLMs) ranging from 0.5B to 1.5B parameters currently underperform on Text-to-SQL tasks due to their limited logical reasoning capabilities. However, SLMs offer inherent advantages in inference speed and suitability for edge deployment. To explore their potential in Text-to-SQL applications, we leverage recent advancements in post-training techniques. Specifically, we used the open-source SynSQL-2.5M dataset to construct two derived datasets: SynSQL-Think-916K for SQL generation and SynSQL-Merge-Think-310K for SQL merge revision. We then applied supervised fine-tuning and reinforcement learning-based post-training to the SLM, followed by inference using a corrective self-consistency approach. Experimental results validate the effectiveness and generalizability of our method, SLM-SQL. On the BIRD development set, the five evaluated models achieved an average improvement of 31.4 points. Notably, the 0.5B model reached 56.87% execution accuracy (EX), while the 1.5B model achieved 67.08% EX. We will release our dataset, model, and code to github: https://github.com/CycloneBoy/slm_sql.
| Model | Base Model | Train Method | Modelscope | HuggingFace |
|---|---|---|---|---|
| SLM-SQL-Base-0.5B | Qwen2.5-Coder-0.5B-Instruct | SFT | 🤖 Modelscope | 🤗 HuggingFace |
| SLM-SQL-0.5B | Qwen2.5-Coder-0.5B-Instruct | SFT + GRPO | 🤖 Modelscope | 🤗 HuggingFace |
| CscSQL-Merge-Qwen2.5-Coder-0.5B-Instruct | Qwen2.5-Coder-0.5B-Instruct | SFT + GRPO | 🤖 Modelscope | 🤗 HuggingFace |
| SLM-SQL-Base-1.5B | Qwen2.5-Coder-1.5B-Instruct | SFT | 🤖 Modelscope | 🤗 HuggingFace |
| SLM-SQL-1.5B | Qwen2.5-Coder-1.5B-Instruct | SFT + GRPO | 🤖 Modelscope | 🤗 HuggingFace |
| CscSQL-Merge-Qwen2.5-Coder-1.5B-Instruct | Qwen2.5-Coder-1.5B-Instruct | SFT + GRPO | 🤖 Modelscope | 🤗 HuggingFace |
| SLM-SQL-Base-0.6B | Qwen3-0.6B | SFT | 🤖 Modelscope | 🤗 HuggingFace |
| SLM-SQL-0.6B | Qwen3-0.6B | SFT + GRPO | 🤖 Modelscope | 🤗 HuggingFace |
| SLM-SQL-Base-1.3B | deepseek-coder-1.3b-instruct | SFT | 🤖 Modelscope | 🤗 HuggingFace |
| SLM-SQL-1.3B | deepseek-coder-1.3b-instruct | SFT + GRPO | 🤖 Modelscope | 🤗 HuggingFace |
| SLM-SQL-Base-1B | Llama-3.2-1B-Instruct | SFT | 🤖 Modelscope | 🤗 HuggingFace |
| Dataset | Modelscope | HuggingFace |
|---|---|---|
| SynsQL-Think-916k | 🤖 Modelscope | 🤗 HuggingFace |
| SynsQL-Merge-Think-310k | 🤖 Modelscope | 🤗 HuggingFace |
| bird train and dev dataset | 🤖 Modelscope | 🤗 HuggingFace |
1
2@misc{sheng2025slmsqlexplorationsmalllanguage,
3 title={SLM-SQL: An Exploration of Small Language Models for Text-to-SQL},
4 author={Lei Sheng and Shuai-Shuai Xu},
5 year={2025},
6 eprint={2507.22478},
7 archivePrefix={arXiv},
8 primaryClass={cs.CL},
9 url={https://arxiv.org/abs/2507.22478},
10}
11
12@misc{sheng2025cscsqlcorrectiveselfconsistencytexttosql,
13 title={CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learning},
14 author={Lei Sheng and Shuai-Shuai Xu},
15 year={2025},
16 eprint={2505.13271},
17 archivePrefix={arXiv},
18 primaryClass={cs.CL},
19 url={https://arxiv.org/abs/2505.13271},
20}