Important Notice: This fine-tuned disease diagnosis model is intended STRICTLY for research and educational purposes. It is NOT CERTIFIED for clinical use and should NOT be used for real medical diagnosis. The model may generate inaccurate or incomplete suggestions based on its training data.
Medical Disclaimer: This AI system is not a substitute for professional medical advice, diagnosis, or treatment. Always seek the guidance of qualified healthcare providers with any questions regarding medical conditions. The developers disclaim all responsibility for any decisions made based on this model's outputs.
Model Description
This model is a LoRA-adapted version of Mistral-7B-v0.3, specifically fine-tuned for disease diagnosis research. Key features:
Architecture: 7B parameter transformer with LoRA adapters (r=32, alpha=64)
Training Approach: 4-bit quantized training with NF4 quantization
Capabilities: Symptom analysis and differential diagnosis suggestion
Context Window: 8k tokens for comprehensive case analysis
Metrics
Metric
Accuracy
Top-1
87%
Top-3
92%
Check out this Notebook for more information about test results.
Training Configuration
Hyperparameters
Parameter
Value
Learning Rate
1e-4
Batch Size
64
Sequence Length
93 tokens
Optimizer
Paged AdamW 32-bit
Scheduler
Cosine with 3% Warmup
LoRA Rank
32
LoRA Alpha
64
Epochs
1
Technical Stack
PEFT: 0.14.0
Transformers: 4.49.0
PyTorch: 2.6.0+cu124
Datasets: 3.3.2
Bitsandbytes: 0.43.0
TRL: 0.8.6
Intended Use
Research Applications:
Medical education simulations
Symptom pattern analysis
Diagnostic decision support systems research
Prohibited Uses:
Clinical diagnosis
Patient treatment decisions
Self-diagnosis tools
Ethical Considerations
Bias Mitigation: Regularization techniques applied during training
Privacy: No real patient data used in training
Transparency: Full architecture details available in configuration
