Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
malaysian-llama-3-8b-262k – AI Model by mesolitica | AlphaNeural AI
You can deploy this model and start earning money today!
mesolitica
/
malaysian-llama-3-8b-262k
like
0
transformers
safetensors
llama
text-generation
autotrain_compatible
text-generation-inference
endpoints_compatible
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Malaysian Llama-3 8B 262144 context length
262144 context length and 207100000 RoPE Theta.
WanDB,
https://wandb.ai/huseinzol05/EasyContext-262144?nw=nwuserhuseinzol05
Source code,
https://github.com/mesolitica/malaya/tree/master/session/llama3#extend-1m-context-length
Special thanks to
https://github.com/jzhang38/EasyContext
for wrapping
https://github.com/zhuzilin/ring-flash-attention
for distributed training!
