Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
zubenelgenubi-124m – AI Model by farpluto | AlphaNeural AI
You can deploy this model and start earning money today!
farpluto
/
zubenelgenubi-124m
like
0
safetensors
gpt2
text-generation
knowledge-distillation
symbolic-reasoning
from-scratch
conversational
en
HuggingFaceFW/fineweb-edu
apache-2.0
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
124M GPT with Symbolic Reasoning Distillation
A
124M-parameter
GPT-2 trained
from scratch
on
FineWeb-Edu
with
knowledge distillation
from
SmolLM-135M-Instruct
.
Component
Value
Parameters
~124M
Layers
12
Heads
12
Embedding dim
768
Context
512
Loss
0.5 CE + 0.5 KL
Hardware
1x A100
Time
~75 min
Tokens
327,680,000
Best loss
326.0111
