Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
llama3-8b-instruct-CLA-2 – AI Model by answerdotai | AlphaNeural AI
You can deploy this model and start earning money today!
answerdotai
/
llama3-8b-instruct-CLA-2
like
0
transformers
llama
text-generation
autotrain_compatible
endpoints_compatible
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Transformers :
https://github.com/huggingface/transformers/blob/47fedc1665f14a60260b1b8357e682669300093a/src/transformers/models/llama/modeling_llama.py
Training Script:
https://github.com/AnswerDotAI/fsdp_qlora/blob/3f7c583e985ff35e37a7b7497a7d4fedb77df695/experiments/cla/train.sh
This model shares KV activations every 2 layers. For example, layer 1 uses layer 0 kv activations, layer 3 uses layer 2 kv activations, etc..
