mistral-7b-medical-instruct – AI Model by AbhijithSK | AlphaNeural AI

You can deploy this model and start earning money today!

AbhijithSK/
mistral-7b-medical-instruct

like

0

medalpaca/medical_meadow_pubmed_causal

mistralai/Mistral-7B-Instruct-v0.2

Views

No views yet

Mistral-7B Medical Instruct (QLoRA Fine-tuned)

Fine-tuned version of mistralai/Mistral-7B-Instruct-v0.2 on medical Q&A data using QLoRA.

Training Results

Metric	Value
Final Train Loss	0.022
Token Accuracy	98.9%
Epochs	3
Training Time	~2h 50min on T4 GPU
Trainable Params	13.6M / 7.25B (0.19%)

Method

Base model: mistralai/Mistral-7B-Instruct-v0.2
Dataset: medalpaca/medical_meadow_pubmed_causal (2,446 samples)
Quantization: 4-bit NF4 (QLoRA)
LoRA rank: 16, alpha: 32
Target modules: q_proj, v_proj, k_proj, o_proj

Usage

from peft import PeftModel from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2") model = PeftModel.from_pretrained(base, "AbhijithSK/mistral-7b-medical-instruct") tokenizer = AutoTokenizer.from_pretrained("AbhijithSK/mistral-7b-medical-instruct")

Limitations

For research only. Not for clinical use. Always consult a qualified physician.

Author

Abhijith SK - GitHub: https://github.com/AbhijithSK