HFA Parameter Mapping Debug Output
Issue
HFA model produces random predictions with extremely low probabilities (~0.000008) despite successful checkpoint loading.
Analysis Date
2025-09-15 04:26:52
Files
hfa_parameter_debug_*.txt: Complete debug outputhfa_parameter_analysis_*.json: Structured analysis data
Problem
The model configuration matches checkpoint dimensions (256 hidden, 8 heads, 6 layers) but parameters aren't loading correctly, resulting in random predictions instead of meaningful language modeling.
Next Steps
- Verify exact parameter name mapping between checkpoint and model
- Check if critical parameters (embedding, attention, lm_head) are being loaded
- Ensure parameter values are being copied correctly (not just shapes)
