Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
smollm-135m-fineweb-icelandic-from-scratch – AI Model by jekunz | AlphaNeural AI
You can deploy this model and start earning money today!
jekunz
/
smollm-135m-fineweb-icelandic-from-scratch
like
0
transformers
safetensors
llama
text-generation
conversational
is
HuggingFaceFW/fineweb-2
apache-2.0
autotrain_compatible
text-generation-inference
endpoints_compatible
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
This is a model with the same specifications as SmolLM2-135M trained from scratch on the Icelandic portion of Fineweb-2. It is intended as a baseline for my research is probably rather bad for most purposes :)
Training:
1 Epoch
Learning rate: 5e-4
LR scheduler: Cosine
Warmup ratio: 0.05
Batch size: 1
4 A100 (40GB) GPUs
Gradient accumulation steps: 64
Effective batch size: 256
Max. context length: 8192 tokens
