Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
TinyMistral-248M-v2 – AI Model by Locutusque | AlphaNeural AI
You can deploy this model and start earning money today!
Locutusque
/
TinyMistral-248M-v2
like
0
transformers
pytorch
safetensors
mistral
text-generation
en
Skylion007/openwebtext
Locutusque/TM-DATA
apache-2.0
autotrain_compatible
text-generation-inference
endpoints_compatible
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Training
This model was trained on two datasets, shown in this model page.
Skylion007/openwebtext: 1,000,000 examples at a batch size of 32-4096 (1 epoch)
Locutusque/TM-DATA: All examples at a batch size of 12288 (3 epochs) Training took approximately 500 GPU hours on a single Titan V.
Metrics
You can look at the training metrics here:
https://wandb.ai/locutusque/TinyMistral-V2/runs/g0rvw6wc
🔥 This model performed excellently on TruthfulQA, outperforming models more than 720x its size. These models include: mistralai/Mixtral-8x7B-v0.1, tiiuae/falcon-180B, berkeley-nest/Starling-LM-7B-alpha, upstage/SOLAR-10.7B-v1.0, and more. 🔥
