Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
llama-161M-100B – AI Model by abacaj | AlphaNeural AI
You can deploy this model and start earning money today!
abacaj
/
llama-161M-100B
like
0
transformers
onnx
safetensors
llama
text-generation
apache-2.0
autotrain_compatible
text-generation-inference
endpoints_compatible
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
llama-161M
Trained on 100B tokens.
1e-3 LR
0.1 wd
WSD scheduler with 10% decay
80% code, 10% NL, 10% instruction data
Dataset decontaminated against popular benchmarks following
bigcode
8x3090s 110~ hours
This is a
base
pretrained model and requires further fine tuning to be useful.
Model Details
openai/openai_humaneval
(greedy)
mbpp
(greedy)
9.2%
9.8%
