Empirical verification of Maximal Update Parametrization (μP): Demonstrating that μP achieves width-invariant gradient updates, enabling hyperparameter transfer across model scales.
μP shows 42.8% less width-dependence than Standard Parametrization (SP) in relative gradient updates, confirming the theoretical prediction that μP enables hyperparameter transfer across model widths.
