Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
smollm-135m-lora-fineweb-icelandic – AI Model by jekunz | AlphaNeural AI
You can deploy this model and start earning money today!
jekunz
/
smollm-135m-lora-fineweb-icelandic
like
0
peft
safetensors
text-generation
conversational
is
HuggingFaceFW/fineweb-2
HuggingFaceTB/SmolLM2-135M-Instruct
adapter
apache-2.0
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
This model is a SmolLM2-135M-Instruct model fine-tuned on the Icelandic portion of Fineweb-2. It is intended for my research and has not been evaluated more broadly yet.
LoRA setup:
Rank: 256
Alpha: 512
Target modules: ["up_proj", "down_proj", "gate_proj", "o_proj"]
Training:
1 Epoch
Learning rate: 8e-4
LR scheduler: Cosine
Warmup ratio: 0.05
Batch size: 1
4 A100 (40GB) GPUs
Gradient accumulation steps: 64
Effective batch size: 256
Max. context length: 8192 tokens
