Does position bias severity depend on the cognitive complexity required? Are "remember" tasks (simple recall) more sensitive to position than "create" tasks (generation)?
| Cognitive Level | Expected Position Sensitivity | Rationale |
|---|---|---|
| Remember | High | Pure retrieval, no reasoning buffer |
| Understand | Moderate-High | Requires comprehension but still lookup-based |
| Apply | Moderate | Must retrieve + transform |
| Analyze | Moderate | Multi-point comparison |
| Evaluate | Low-Moderate | Uses criteria across context |
| Create | Low | Generation is less position-dependent |
1pip install -r requirements.txt
2
3# Run all 6 cognitive levels
4python run_all.py --model Qwen/Qwen2.5-1.5B-Instruct --num-examples 50
5
6# Run specific levels
7python run_all.py --levels remember apply evaluate"Position Bias Index (PBI) correlates negatively with cognitive complexity: PBI_remember = 0.42, PBI_create = 0.12 (p < 0.001). Higher cognitive demands partially mitigate positional bias through attention redistribution."
1@software{cognitive_hierarchy_position_bias,
2 title={Cognitive Hierarchy Position Bias: Bloom's Taxonomy Evaluation of Long-Context Models},
3 author={abhshkp},
4 year={2026},
5 url={https://huggingface.co/abhshkp/cognitive-hierarchy-position-bias}
6}