recursive-sat-qwen2.5-1.5b
This is a paper model: the REC-3 release artifact from a paper-aligned replication of recursive SAT reasoning at 1.5B scale.
It is a supervised fine-tune of Qwen/Qwen2.5-1.5B-Instruct trained on recursive SAT traces derived from SATBench with explicit <call> / <return> structure. The goal is research replication and analysis, not general-purpose production use.
What This Model Is
- Base model:
Qwen/Qwen2.5-1.5B-Instruct
- Release artifact:
results/runs/REC-3/published_model
- Training run:
REC-3
- Seed:
303
- Config:
configs/rec_seed303.yaml
- Dataset source:
LLM4Code/SATBench
- Task: SAT / UNSAT classification via recursive trace supervision
Why REC-3
REC-1 and REC-3 tie on mean accuracy, but REC-3 is the cleaner release candidate on end-to-end behavior:
- Mean accuracy:
45.33%
- Easy:
39.0%
- Medium:
54.0%
- Hard:
43.0%
- Parse failure rate:
7.0%
- Valid trace rate:
99.0%
Compared with REC-1, REC-3 keeps the same mean accuracy while reducing parse failure (7.0% vs 8.33%), improving hard accuracy (43.0% vs 42.0%), and slightly improving valid trace rate (99.0% vs 98.33%).
Important Caveat
This is a paper model, not a claim of robust general recursive reasoning.
The underlying paper draft treats the result as a qualified replication:
- recursive SFT improves end-to-end SATBench accuracy over raw direct prompting
- the strongest gain is on medium-difficulty SAT instances
- absolute performance remains far below the 3B source-paper result
- recursion behavior is still shallow overall
Use this release as a research artifact tied to the experiment, metrics, and discussion in the paper repo.
Training Summary
- Objective:
recursive_sft
- Train examples:
74,827
- Validation examples:
619
- Global step:
46,770
- Best checkpoint:
checkpoint-9354
- Accelerator used for the main run:
cuda
Evaluation Summary
Main held-out evaluation uses 100 examples each from SATBench easy, medium, and hard buckets.
Baseline vs released model:
- Base direct prompt mean accuracy:
37.33%
REC-3 mean accuracy: 45.33%- Absolute gain:
+8.0 points
- Base parse failure rate:
28.67%
REC-3 parse failure rate: 7.0%
Prompt Format
The model was trained on recursive traces using:
<call> ... </call> for subproblem decomposition<return> ... </return> for compact returned answers
It is best treated as a specialized research model for this protocolized SAT setting.
Files In This Release
model.safetensorsconfig.jsongeneration_config.jsontokenizer.jsontokenizer_config.jsonchat_template.jinjaexport_metadata.json
Intended Use
- paper artifact release
- replication reference
- SAT recursive-trace evaluation
- qualitative inspection of recursive protocol behavior
Out Of Scope
- production reasoning system
- general mathematical reasoning benchmark model
- safety-critical use
- claims beyond the SATBench replication setting
