You can deploy this model and start earning money today!

AETHORIA-AI/
TR-HASH-MoE-200M-130B-Checkpoints

Name: TR-HASH-MoE-200M-130B-Checkpoints
Availability: InStock

Views

No views yet

TR-HASH MoE 200M — 70B Unique / 130B Replay Checkpoints

Status: training complete. This is the raw checkpoint backup repository, not the finished model release. It exists as a safety net and a record of the training trajectory — token-pack folders include model weights and resumable optimizer/scheduler state.

Checkpoints are uploaded automatically at token-pack boundaries and on clean/interrupted shutdown by scripts/sync_checkpoints_to_hf.py.
Folder names follow {tag}_{step} (token_pack_NNN_STEP, final_STEP, interrupted_STEP).
The PIQA sweep below is exploratory, not a full validated evaluation suite. Do not treat an individual checkpoint as a finished model release.
The architecture config is tracked at model_config.yaml in this repo (not embedded in checkpoint.pt).

Exploratory zero-shot checks (not a full evaluation)

Informal checks across the final token-pack checkpoints, via scripts/convert_to_mlx.py + scripts/eval_mlx_zero_shot.py, zero-shot causal-log-likelihood scoring, no chat template. Every row uses the same tokenizer, MLX FP16 inference path, PIQA validation split, and 1,838 examples:

Checkpoint	Tokens trained	Learning rate	PIQA acc	PIQA acc_norm
`token_pack_032_132239`	104.00B	3.00e-4	0.6736	0.6763
`token_pack_033_136371`	107.25B	2.91e-4	0.6774	0.6746
`token_pack_034_140504`	110.50B	2.62e-4	0.6768	0.6703
`token_pack_035_144636`	113.75B	2.19e-4	0.6741	0.6801
`token_pack_036_148769`	117.00B	1.67e-4	0.6768	0.6774
`token_pack_037_152901`	120.25B	1.14e-4	0.6676	0.6697
`token_pack_038_157034`	123.50B	7.01e-5	0.6578	0.6638
`token_pack_039_161166`	126.75B	4.04e-5	0.6567	0.6529
`token_pack_040_165298` / final	130.00B	3.00e-5	0.6545	0.6561

token_pack_035_144636 has the highest acc_norm in this sweep. The trajectory is not monotonic, and PIQA alone must not be used as a complete model-quality or checkpoint-selection criterion.

The finished base model is published separately as AETHORIA-AI/TR-HASH-MoE-200M-130B.

See Complexity Framework for the training code.

AETHORIA-AI/TR-HASH-MoE-200M-130B-Checkpoints

TR-HASH MoE 200M — 70B Unique / 130B Replay Checkpoints

Exploratory zero-shot checks (not a full evaluation)

AETHORIA-AI/
TR-HASH-MoE-200M-130B-Checkpoints