Beta
Explore
Marketplace
Neural Labs
Playground
Wallet
Docs
mistral-7b-alpaca-sft – AI Model by CorticalStack | AlphaNeural AI
You can deploy this model and start earning money today!
CorticalStack
/
mistral-7b-alpaca-sft
like
0
transformers
safetensors
mistral
text-generation
apache-2.0
autotrain_compatible
text-generation-inference
endpoints_compatible
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
mistral-7b-alpaca-sft
mistral-7b-alpaca-sft is an SFT fine-tuned version of
unsloth/mistral-7b-bnb-4bit
using the
yahma/alpaca-cleaned
dataset.
Fine-tuning configuration
LoRA
r: 256
LoRA alpha: 128
LoRA dropout: 0.0
Training arguments
Epochs: 1
Batch size: 4
Gradient accumulation steps: 6
Optimizer: adamw_torch_fused
Max steps: 100
Learning rate: 0.0002
Weight decay: 0.1
Learning rate scheduler type: linear
Max seq length: 2048
4-bit bnb: True
Trained with
Unsloth
and Huggingface's TRL library.
