SQL Socratic Models are a collection of fine-tuned large language models designed for Socratic SQL instruction in higher education. Unlike standard Text-to-SQL systems, these models are trained to guide learners through reasoning steps without producing final SQL solutions, supporting conceptual understanding and active learning in STEM contexts.

Supported architectures:

Phi-3
Qwen2.5
Gemma2

Intended Use

These models are designed for:

Teaching SQL concepts in higher education
Supporting STEM learners through guided reasoning
Providing step-by-step Socratic hints for SQL problems
Assisting debugging and conceptual clarification

Important Constraint

The models are intentionally trained to:

✅ Provide reasoning steps and conceptual hints
❌ Avoid generating complete SQL solutions

This ensures alignment with pedagogical goals such as scaffolding and learner engagement.

Training Data: SQL-Instruct Corpus

We construct SQL-Instruct, a domain-specific Socratic instruction corpus, by mining high-quality interactions from Stack Overflow. This platform captures real-world misconceptions, debugging challenges, and conceptual gaps encountered by learners and practitioners.

Data Collection

To ensure high-quality instructional signals, we filter SQL-tagged questions based on community impact. The resulting dataset has:

1.27 billion total views
128,535 average views per question

For each selected entry, we extract:

Problem descriptions
User-submitted SQL attempts
Executable SQL from accepted solutions

This yields 9,916 unique questions.

Socratic Augmentation

Each example is transformed into a Socratic instructional format using GPT-4o, which generates:

Guided reasoning steps
Conceptual hints
Question decomposition

This ensures the dataset emphasizes instructional scaffolding rather than answer generation.

Dataset Composition

Intermediate questions: 8,604
Advanced questions: 629
Debugging tasks: 531

The dataset emphasizes challenging reasoning scenarios, particularly:

JOIN operations
Aggregations and grouping
Query optimization

We further ensure reliability by selecting entries with a median Stack Overflow score of 27.

Training Procedure

Phase 2: Fine-Tuning

We apply Full Fine-Tuning (FFT) on small, open-source LLMs under pedagogical constraints designed to:

Encourage conceptual scaffolding
Promote step-by-step reasoning
Discourage direct SQL answer generation

Evaluation

Phase 3 Metrics

Models are evaluated using:

BERTScore → semantic alignment with expected reasoning
ROUGE-L → detection of answer leakage (i.e., unintended full SQL generation)

Key Contributions

Socratic SQL instruction tuning for higher education
SQL-Instruct dataset derived from real-world misconceptions
Multi-model fine-tuning across Phi-3, Qwen2.5, and Gemma2
Evaluation framework balancing reasoning quality and answer leakage
Ablation study identifying factors enabling:
- Misconception-based feedback
- Iterative guidance
- Instructor-like reasoning behavior

Limitations

Models may still occasionally generate partial SQL fragments
Evaluation focuses on semantic similarity rather than full pedagogical outcomes
Dataset is derived from Stack Overflow and may reflect community biases

Ethical Considerations

These models are designed to support learning, not replace it. By avoiding full solution generation, they aim to:

Encourage critical thinking
Reduce over-reliance on AI-generated answers
Support equitable access to SQL learning resources

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("sriram882004/SQL-Socratic-Models/phi3")
tokenizer = AutoTokenizer.from_pretrained("sriram882004/SQL-Socratic-Models/phi3")