Qwen2.5-7B-Instruct-Medical
Model Description
Qwen2.5-7B-Instruct-Medical is a medical domain-specific model fine-tuned from the Qwen2.5-7B-Instruct model using 340,000 medical dialogue samples. This model is optimized to provide accurate and contextually relevant responses to medical-related inquiries, making it an excellent choice for healthcare applications such as medical chatbots, decision support systems, and educational tools.
Model Details
- Base Model: Qwen2.5-7B-Instruct
- Fine-tuning Dataset: 340,000 medical dialogue samples
- Training Duration: 51 hours
- Hardware Used: 6x NVIDIA RTX 3090 (24GB VRAM)
- Optimization Algorithm: AdamW
- Training Method: LoRA (Low-Rank Adaptation)
- Training Framework: PyTorch
Performance
- BLEU-4 Score:
- Base Model: 23.5 (on a test set of 500 samples)
- Fine-tuned Model: 55.7 (on the same test set)
This shows a significant improvement in the model's ability to generate more fluent and contextually relevant responses after fine-tuning on the medical dialogue dataset.
Intended Use
This model is specifically tailored for medical dialogue tasks and can be used for:
- Medical question answering
- Healthcare chatbots
- Clinical decision support systems
- Medical education and training
Performance
The model exhibits a strong understanding of medical terminology, clinical contexts, and patient interactions, making it a powerful tool for applications in healthcare and medical research.
Usage
To use this model, you can load it using the transformers library in Python:
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model = AutoModelForCausalLM.from_pretrained("path_to_model")
4tokenizer = AutoTokenizer.from_pretrained("path_to_model")
5
6input_text = "What are the symptoms of diabetes?"
7inputs = tokenizer(input_text, return_tensors="pt")
8output = model.generate(**inputs)
9print(tokenizer.decode(output[0], skip_special_tokens=True))
10<
Limitations
While this model has been fine-tuned on a medical dialogue dataset, it may still make errors or provide inaccurate responses in highly specialized medical domains or cases where the input data falls outside the training data's coverage. Always ensure human supervision in critical healthcare scenarios.
License
This model is released under the MIT License.
Acknowledgements
- Dataset: 340,000 medical dialogues (From Modelscope).
- LoRA (Low-Rank Adaptation): This technique was used to efficiently fine-tune the model without modifying the full parameter set, allowing for faster and more memory-efficient training.
