A fine-tuned Mistral-7B-Instruct-v0.3 model specifically trained for generating medical rationales and explanations.
The model was trained using QLoRA on a custom dataset of medical rationales.
Model Details
Model Description
This model is a fine-tuned version of Mistral-7B-Instruct-v0.3, specifically optimized for generating detailed medical rationales and explanations.
It is mainly intended to be used in METEORA Rerankers of medical RAG systems.
It was trained using Low-Rank Adaptation (LoRA) on a dataset of medical reasoning tasks, resulting in an 80%+ improvement in performance metrics compared to the base model.
Developed by: Chidiebere Okoene
Model type: Causal Language Model (Decoder-only)
Language(s) (NLP): English
License: MIT
Finetuned from model: mistralai/Mistral-7B-Instruct-v0.3
This model is intended for generating medical rationales, explanations, and reasoning for healthcare-related queries. It can be used by:
Medical educators creating teaching materials
Healthcare professionals seeking second opinions or explanations
Medical students learning diagnostic reasoning
Researchers exploring medical AI applications
Downstream Use [optional]
This model can be integrated into:
METEORA Reranker for Medical RAG systems
Clinical decision support systems
Healthcare chatbots for patient education
Medical documentation assistants
Out-of-Scope Use
This model should not be used for:
Direct patient diagnosis without human supervision
Making treatment decisions without clinical validation
Replacing licensed medical professionals
Generating medical advice for serious conditions
Bias, Risks, and Limitations
Training Data Bias: The model was trained on a specific dataset of medical rationales and may not cover all medical specialties or rare conditions
Accuracy Limitations: While performance improved significantly, the model may still generate incorrect or incomplete information
Temporal Limitations: Medical knowledge evolves rapidly, and the model may not reflect the latest guidelines or research
Demographic Biases: The training data may not adequately represent all patient populations
Recommendations
Always verify model outputs with current medical literature and guidelines
Use this model as an educational tool rather than a diagnostic tool
Implement human oversight for any clinical applications
Regularly update the model with new medical knowledge
Disclose the AI-assisted nature of generated content to end users
How to Get Started with the Model
Use the code below to get started with the model.
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
2import torch
34model_name ="chidiokoene/mistral-7b-med-rationales-finetuned"56# Load model and tokenizer7model = AutoModelForCausalLM.from_pretrained(8 model_name,9 device_map="auto",10 torch_dtype=torch.float16
11)12tokenizer = AutoTokenizer.from_pretrained(model_name)1314# Generate rationales15defgenerate_rationale(prompt):16 inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=512)17 inputs ={k: v.to(model.device)for k, v in inputs.items()}1819with torch.no_grad():20 outputs = model.generate(21**inputs,22 max_new_tokens=256,23 temperature=0.7,24 do_sample=True,25 pad_token_id=tokenizer.eos_token_id
26)2728return tokenizer.decode(outputs[0], skip_special_tokens=True)2930# Example usage31prompt = "Given the user query below, generate 3 concise rationales (1–2 sentences each) describing what evidence a correct passage should contain.32 Explain the mechanism of action of metformin intype2 diabetes."
33rationale = generate_rationale(prompt)34print(rationale)
Training Details
Training Data
The model was fine-tuned on a proprietary dataset of medical rationales containing approximately 11,362 training examples and 3,246 validation examples.
The data consisted of medical questions paired with detailed explanatory rationales.
Training Procedure
Preprocessing [optional]
Text was tokenized using the Mistral tokenizer
Sequences were truncated or padded to 1024 tokens
Special tokens were added for instruction following
Training time: ~13 hours on a single GPU with 15GB VRAM
Model size: ~15GB (4-bit quantized)
Inference speed: ~2.9 samples/second
Evaluation
Testing Data, Factors & Metrics
Testing Data
The model was evaluated on a held-out validation set of 1,624 medical rationale examples.
Factors
[More Information Needed]
Metrics
Perplexity (lower is better)
Average cross-entropy loss (lower is better)
Inference speed (samples per second)
Results
Metric Baseline Model Fine-tuned Model Improvement
Perplexity 7.78 1.51 80.6%
Average Loss 2.05 0.41 79.9%
Inference Speed 5.17 samples/sec 2.91 samples/sec -43.7%
The fine-tuned model shows exceptional improvement in understanding and generating medical rationales,
with over 80% improvement in both perplexity and loss metrics. The reduction in inference speed is expected due to the added LoRA parameters.
The fine-tuning process was highly successful, resulting in a model that significantly outperforms the base Mistral-7B model on medical rationale generation tasks while maintaining reasonable inference speed.
