Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
CorticalStack_-_mistral-7b-openhermes-2.5-sft-4bits – AI Model by RichardErkhov | AlphaNeural AI
You can deploy this model and start earning money today!
RichardErkhov
/
CorticalStack_-_mistral-7b-openhermes-2.5-sft-4bits
like
0
safetensors
mistral
4-bit
bitsandbytes
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Quantization made by Richard Erkhov.
Github
Discord
Request more models
mistral-7b-openhermes-2.5-sft - bnb 4bits
Model creator:
https://huggingface.co/CorticalStack/
Original model:
https://huggingface.co/CorticalStack/mistral-7b-openhermes-2.5-sft/
Original model description:
license: apache-2.0
mistral-7b-openhermes-2.5-sft
mistral-7b-openhermes-2.5-sft is an SFT fine-tuned version of
unsloth/mistral-7b-bnb-4bit
using the
teknium/OpenHermes-2.5
dataset.
Fine-tuning configuration
LoRA
r: 256
LoRA alpha: 128
LoRA dropout: 0.0
Training arguments
Epochs: 1
Batch size: 4
Gradient accumulation steps: 6
Optimizer: adamw_torch_fused
Max steps: 100
Learning rate: 0.0002
Weight decay: 0.1
Learning rate scheduler type: linear
Max seq length: 2048
4-bit bnb: True
Trained with
Unsloth
and Huggingface's TRL library.
