Qwen/Qwen2.5-Math-7B post-trained with Evolution Strategies (ES) for mathematical reasoning, from Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategies.

ES is a population-based, gradient-free method that optimises directly in weight space through random perturbations. Unlike RL, which sharpens the output distribution around its single best guess, ES raises pass@1 while preserving solution coverage — so the model keeps benefiting from additional test-time samples. See the paper repository.

Training setting

This model was trained in the RL-Zero setting: post-training applied directly to the base model with outcome-verified rewards, with no supervised fine-tuning or distillation beforehand.

Data — the MATH dataset restricted to levels 3–5 (Hendrycks et al., 2021; Liu et al., 2025)
Reward — binary, verifier-checked correctness of the final \boxed{} answer
Maximum response length during training — 3,000 tokens

Training parameters

As reported in the paper appendix (Table 2):

Hyperparameter	Value
Perturbation scale σ	0.001
Learning rate α	0.0005 (σ / 2)
Population size	32
Train batch size	512
Max response length	3,000
Rollout temperature	0.0
Total training steps	500

Evaluation

Following Yue et al. (2025), all benchmarks are sampled at:

Parameter	Value
Temperature	0.6
Top-p	0.95
Max tokens	16,384

Reproduce with the evaluation harness:

bash

python -m es_pass_k.math_pass_k \
  --model-path conorfhayes/Qwen2.5-Math-7B-ES-MATH \
  --eval-dataset datasets/evaluation_suite/math/math500 \
  --k 256 --temp 0.6 --top-p 0.95 --max-tokens 16384 \
  --template math \
  --use-gpus 0

Usage

The model expects the Qwen2.5-Math prompt format, with the reasoning instruction in the system turn. Using a different template is out-of-distribution and will depress accuracy:

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("conorfhayes/Qwen2.5-Math-7B-ES-MATH", torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained("conorfhayes/Qwen2.5-Math-7B-ES-MATH")

question = "What is the smallest positive integer n such that n! is divisible by 1000?"
prompt = (
    "<|im_start|>system\nPlease reason step by step, and put your final answer "
    "within \\boxed{}.<|im_end|>\n<|im_start|>user\n"
    + question
    + "<|im_end|>\n<|im_start|>assistant\n"
)

out = model.generate(**tok(prompt, return_tensors="pt").to(model.device),
                     max_new_tokens=2048, temperature=0.6, top_p=0.95, do_sample=True)
print(tok.decode(out[0], skip_special_tokens=True))

Sampling matters: this model is intended for repeated sampling, where its broader output distribution is the advantage. Greedy decoding discards exactly the property ES optimises for.

Limitations

Trained and evaluated only on English mathematical reasoning. Behaviour on other domains, languages, or task formats is uncharacterised.
Inherits the capabilities and biases of Qwen/Qwen2.5-Math-7B.
No safety alignment was performed. This is a research artefact for studying solution coverage under test-time scaling, not a deployment-ready assistant.
Reward was outcome-based only, so correct answers reached by faulty reasoning were rewarded.

Citation

bibtex

@article{hayes2026beyond,
  title   = {Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategies},
  author  = {Hayes, Conor F. and Meyerson, Elliot and Schweighofer, Kajetan and
             Dailey, Roberto and Hodjat, Babak and Miikkulainen, Risto and Qiu, Xin},
  year    = {2026}
}