E2E 3B p24-r1: full_network_binary_router
This is a weights-only, inference-ready Orbax release of the terminal step-1024 E2E checkpoint.
- Hidden size: 2560
- Layers: 32
- Training sequence length: 8192
- Router z coefficient: 0.0001
- GTCF: true
- Evaluation: NIAH-1: complete, NIAH-2: complete, NIAH-3: complete, LongBenchV2: complete, PG-19: pending_by_policy
PG-19: pending_by_policy is an explicit status, not a numeric score.
Use checkpoint/ as the manager root and restore step 1024 through the project's read-only routed inference loader. The package includes only model_weights; it cannot resume optimizer training.
No tokenizer bytes, optimizer state, data-iterator state, tracking metadata, internal service paths, or raw evaluation samples are included. Use a separately obtained compatible Llama-3-base tokenizer. See LICENSE_NOTICE.md before use or redistribution.
