NousResearch/

eval-Hermes-4-405B-reasoning

	405b-e3-40k-reasoning Evaluation Results




	
	Summary




	

Benchmark Score Metric Samples Overlong rate

aime24 0.819 math_pass@1:64_samples 64 5.6%

aime25 0.781 math_pass@1:64_samples 64 5.3%

arenahard 0.937 eval/overall_winrate 500 0.0%

bbh_generative 0.863 extractive_match 1 4.7%

creative-writing-v3 0.793 creative_writing_score 96 0.0%

drop_generative_nous 0.835 drop_acc 1 1.6%

eqbench3 0.855 eqbench_score 135 0.0%

gpqa_diamond 0.706… See the full description on the dataset page: https://huggingface.co/datasets/NousResearch/eval-Hermes-4-405B-reasoning.