Agent/Programming models were designed to solve mathematical problems by integrating text-based reasoning with code blocks executed by Python interpreter. The models were trained on python-edu-cleaned, a math instruction tuning dataset with 7.6M problem-solutions:

Best performance on tasks that resemble the training data
Better Python explanations
Calibrated idiomatic Python code
Better handling of educational programming prompts
Improved debugging advice
More consistent code formatting

We did not reduce the quality of the model and for some reason did not use quantization for code quality.

	greedy	majority@50
model	GSM8K	MATH	GMS8K	MATH
Shirdel-Agent-4B	65.9	53.6	94.8	65.6
Shirdel-Agent-4B	90.2	54.5	96.9	67.2
Shirdel-Agent-4B	88.8	55.5	96.8	67.6
Shirdel-Agent-4B	90.7	58.3	98.0	70.2
Shirdel-Agent-4B	94.7	56.3	99.1	68.3
Shirdel-Agent-4B	94.6	60.7	99.8	70.4

The pipeline we used to produce these models is fully open-sourced.

Developed by: Tesleum

Tesleum/shirdel-agent-4b

Shirdel-Agent-4B

Tesleum/
shirdel-agent-4b