Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
zubenelgenubi-1.1-124m – AI Model by farpluto | AlphaNeural AI
You can deploy this model and start earning money today!
farpluto
/
zubenelgenubi-1.1-124m
like
0
safetensors
gpt2
text-generation
knowledge-distillation
symbolic-reasoning
chain-of-thought
from-scratch
conversational
en
HuggingFaceFW/fineweb-edu
openai/gsm8k
apache-2.0
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
124M GPT with Symbolic Reasoning Distillation
Trained
from scratch
on mixed data with
dual-alpha distillation
:
Stream
Dataset
Alpha
Purpose
General
FineWeb-Edu
0.2
Language modeling, light teacher guidance
Reasoning
GSM8K chain-of-thought
0.8
Heavy distillation: teacher guides step-by-step math reasoning
Teacher
: SmolLM-135M-Instruct (frozen)
Time
: ~75 min on 1x A100
Tokens
: 327,680,000 (0 reasoning / 20,000 general batches)
Best loss
: 186.6474
