Model Summary
adaption_llama_3_3_70b_instru_adapted_medqa_usmle_in_k_82b0c1ba is a supervised fine-tuned (SFT) medical language model based on Meta Llama 3.3 70B Instruct. The model was adapted using Low-Rank Adaptation (LoRA) on a multilingual medical instruction dataset consisting of clinical vignettes in English and Kazakh.
The model is designed to improve instruction following, clinical reasoning, and medical question answering. Training examples include enhanced prompts, high-quality completions, and reasoning traces, enabling the model to generate more structured and informative medical responses.
Usage
This model can be used for:
- Medical question answering
- Clinical vignette analysis
- Medical instruction following
- Clinical reasoning assistance
- Multilingual medical NLP research
Input
Natural language medical instructions or clinical case descriptions.
Output
Structured medical responses including diagnostic reasoning and treatment recommendations.
Known Limitations
- The model should not be used as a substitute for professional medical judgment.
- Outputs may contain factual inaccuracies or hallucinations.
- Performance may decrease on medical domains not represented in the training data.
- The model is intended for research and educational purposes.
System
This is a standalone causal language model.
Input requirements:
- Plain text prompts
- Clinical vignettes
- Medical questions
- Instruction-following tasks
The generated outputs may be integrated into downstream applications such as:
- Medical assistants
- Clinical education tools
- Question answering systems
- Medical NLP pipelines
Implementation Requirements
Training Hardware
The model was fine-tuned using the Adaption platform on GPU-based infrastructure.
Software
- PyTorch
- Hugging Face Transformers
- PEFT (LoRA)
- Supervised Fine-Tuning (SFT)
Compute Requirements
Training configuration:
- Base model: 70B parameters
- Training method: Supervised Fine-Tuning (SFT)
- LoRA adaptation
- Chat-formatted dataset
Inference requires multiple high-memory GPUs or equivalent accelerated hardware capable of serving a 70B parameter model.
Model Characteristics
Model Initialization
The model was initialized from:
meta-llama/Llama-3.3-70B-Instruct-Reference
It was fine-tuned using LoRA rather than trained from scratch.
Model Stats
| Property | Value |
|---|
| Base Model | Meta Llama 3.3 70B Instruct |
| Parameters | 70B |
| Fine-tuning | LoRA |
| Training Method | SFT |
| Data Format | Chat |
| LoRA Rank | 32 |
| LoRA Alpha | 64 |
| LoRA Dropout | 0 |
| Learning Rate | 1e-4 |
| Epochs | 3 |
| Scheduler | Cosine |
| Warmup Ratio | 0.05 |
| Weight Decay | 0.01 |
Other Details
- LoRA adaptation on all linear layers
- No model pruning
- No quantization applied during training
- No differential privacy techniques were used
Data Overview
The model was trained on a multilingual medical instruction dataset containing 20,352 clinical examples.
The dataset contains approximately:
Each sample contains:
- question
- answer
- enhanced_prompt
- enhanced_completion
- reasoning_trace
- answer_idx
- meta_info
- options
Training Data
Training data consists of clinical vignettes covering multiple medical specialties including:
- Internal Medicine
- Pediatrics
- Psychiatry
- Cardiology
- Neurology
- Endocrinology
- Emergency Medicine
- Surgery
- Infectious Diseases
- Obstetrics and Gynecology
The data were converted into chat format and enhanced through prompt engineering to improve instruction-following behavior and reasoning quality.
Demographic Groups
The dataset contains clinical scenarios involving patients of different ages, sexes, and medical conditions. It does not contain personally identifiable information and is intended for educational and research purposes.
Evaluation Data
The model was evaluated during training using periodic validation checkpoints.
Training configuration:
- Epochs: 3
- Evaluation checkpoints: 5
No separate public benchmark results are included with this release.
Evaluation Results
Summary
The fine-tuned model demonstrated improved instruction following and more structured clinical reasoning compared with the base model on internal validation data.
Subgroup Evaluation Results
No formal subgroup evaluation was conducted.
Performance may vary across:
- Medical specialties
- Languages
- Question complexity
Further evaluation is recommended before deployment in real-world clinical environments.
Fairness
No dedicated fairness evaluation was performed.
The bilingual training data were designed to improve performance across both English and Kazakh medical questions, but performance disparities may still exist.
Usage Limitations
This model is intended for:
- Research
- Education
- Benchmarking
- Medical NLP development
It should not be used as the sole basis for clinical decision-making, diagnosis, or treatment.
Users should independently verify all medical recommendations.
Ethics
The model was developed for research into multilingual medical language models.
Potential risks include:
- Hallucinated medical information
- Incorrect clinical recommendations
- Misinterpretation of ambiguous cases
Users should ensure that outputs are reviewed by qualified healthcare professionals before being applied in clinical settings.
