medquad-lora-r32-best-v2 – AI Model by CHF0101 | AlphaNeural AI

You can deploy this model and start earning money today!

CHF0101/
medquad-lora-r32-best-v2

like

0

Views

No views yet

MedQuAD LoRA r=32

Configuración

Base: mistralai/Mistral-7B-Instruct-v0.3
LoRA r: 32
Módulos: q_proj, k_proj, v_proj
4-bit NF4
Early Stopping: patience=3

Entrenamiento

Training logs (manual, Epoch estimado, 150 max steps):

Step	Epoch	Training Loss	Validation Loss
50	0.023	0.882800	0.845063
100	0.046	0.805700	0.784524
150	0.070	0.754400	0.767385

Evaluación

BertScore F1 (sample test 50): 0.8452

Uso

python

from peft import PeftModel
from transformers import AutoModelForCausalLM
base = AutoModelForCausalLM.from_pretrained('mistralai/Mistral-7B-Instruct-v0.3', load_in_4bit=True)
model = PeftModel.from_pretrained(base, 'CHF0101/medquad-lora-r32-best-v2')