Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
sqlctx-llama2-7b-lora-16 – AI Model by abcdabcd987 | AlphaNeural AI
You can deploy this model and start earning money today!
abcdabcd987
/
sqlctx-llama2-7b-lora-16
like
0
punica
llama-factory
lora
generated_from_trainer
text2text-generation
en
b-mc2/sql-create-context
2310.18547
meta-llama/Llama-2-7b-hf
adapter
apache-2.0
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Punica
Punica: Serving multiple LoRA finetuned LLMs at the cost of one
Paper:
https://arxiv.org/abs/2310.18547
See
https://github.com/punica-ai/punica/tree/master/examples/finetune
Model
Base Model:
Llama-2-7b-hf
LoRA target:
q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
LoRA rank: 16
Training hyperparameters
The following hyperparameters were used during training:
learning_rate: 5e-05
train_batch_size: 32
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
num_epochs: 4.0
Framework versions
Transformers 4.34.1
Pytorch 2.2.0.dev20230911+cu121
Datasets 2.14.4
Tokenizers 0.14.1
