reasoning-degeneration-dev/

t1-full-q3-30b-a3b-5arg

Full Semantic Knowledge Enhanced evaluation: base eval, knowledge synthesis, and enhanced+heldout evaluation.

	Performance Comparison





	

Metric Base (original) Enhanced: Original 5-arg Enhanced: Heldout 5-arg Enhanced: Easier 4-arg Enhanced: Harder 6-arg

pass@1 0.7400 (74.0%) 0.6200 (62.0%) 0.5800 (58.0%) 0.9000 (90.0%) 0.5000 (50.0%)

pass@2 0.7800 (78.0%) 0.7400 (74.0%) 0.6600 (66.0%) 0.9800 (98.0%) 0.5600 (56.0%)

pass@3 0.7800 (78.0%)… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-degeneration-dev/t1-full-q3-30b-a3b-5arg.