Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
MoE-model-test-kaggle – AI Model by wop | AlphaNeural AI
You can deploy this model and start earning money today!
wop
/
MoE-model-test-kaggle
like
0
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
MoE pretraining run
This repository is updated by a Kaggle notebook that resumes from the last saved state.
Status
run_index:
0
source_cursor:
0
global_steps:
0
completed_runs:
0
finished_window:
False
Window policy
rows selected this run:
10000
raw rows scanned this run:
25000
after each completed window, the notebook saves the full state and continues from the next raw cursor on the next trigger
Model shape
layers:
6
d_model:
512
heads:
8
experts:
100
active experts per token:
12
expert hidden size:
768
Recent stats
phase:
bootstrap
Notes
tokenizer: custom byte-level BPE
resume: optimizer, scheduler, RNG, config, tokenizer, model weights, and cursor state are all saved
