Views
No views yet
| Benchmark | Predicted Score |
|---|---|
| GSM8K | 88.3% |
| HumanEval | 59.5% |
| Model | GSM8K | HumanEval |
|---|---|---|
| Qwen2.5-7B-Instruct | 85.4% | 70.1% |
| Qwen2.5-Coder-7B-Instruct | 60.4% | 88.4% |
| Qwen2.5-Math-7B-Instruct | 90.3% | 52.4% |
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model = AutoModelForCausalLM.from_pretrained("researchaudio/Qwen2.5-7B-MathSoup")
4tokenizer = AutoTokenizer.from_pretrained("researchaudio/Qwen2.5-7B-MathSoup")
5
6messages = [{"role": "user", "content": "Solve: What is 15% of 80?"}]
7text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
8inputs = tokenizer(text, return_tensors="pt")
9outputs = model.generate(**inputs, max_new_tokens=256)
10print(tokenizer.decode(outputs[0]))1@misc{soupermodel2025,
2 title={Souper-Model: How Simple Arithmetic Unlocks State-of-the-Art LLM Performance},
3 author={Shalini Maiti and others},
4 year={2025},
5 url={https://arxiv.org/abs/2511.13254},
6}