The Mistral 7b (instruct) 4 bit model is fine-tuned with parameter efficient fine-tuning (PEFT) method of low ranks adaptation (LoRA) of to answer medical Q&A. The model is finetuned using a medical Q&A (instruct) datsest ruslanmv/ai-medical-chatbot with more than 257K data points (question and answer pairs).

This model can be used with a medical or helathcare Q&A chatbot. However, it is advised that to include appropriate system message for best performance. Please look at the usage example below.

Important Info:

Base Model: unsloth/mistral-7b-instruct-v0.3-bnb-4bit
Dataset: ruslanmv/ai-medical-chatbot
Fine-tuning: LoRA via unsloth framework
Quantization: 4-bit (bnb), suitable for GPU inference on consumer-grade hardware

Usage Example

This model is accessible through the Hugging Face Transformers library. First install it using pip: pip install transformers

Use the following sample cod to use and interact with the Medical-Chatbot-Mistral-7B-4bit. Note that the following code is for when running with GPUs.

python

from transformers import pipeline, AutoModelForCausalLM, AutoTokenizer
import torch

 
# load tokenizer and model
model = AutoModelForCausalLM.from_pretrained("Dashanka/medical-chatbot-mistral-7B-instruct-v0.3-bnb-4bit", torch_dtype=torch.bfloat16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Dashanka/medical-chatbot-mistral-7B-instruct-v0.3-bnb-4bit")

# setup the pipeline
generator_pipe = pipeline("text-generation", 
                          model=model, 
                          tokenizer=tokenizer,
                          device_map="auto", # key for multi-GPU/quantized setup
                          torch_dtype=torch.bfloat16
                          )

# Create messages structured for the chat template
sys_message = ''' 
    You are an AI Medical Assistant trained on a dataset of health information. Please be thorough and provide an informative but consise answer. If you don't know the answer to a specific medical inquiry, advise seeking professional help.
    ''' 
user_query = "I get stomach pain when I eat spicy food."

prompt = f"### System: {sys_message}\n### User: {user_query}\n### Assistant:"

response = generator_pipe(prompt, max_new_tokens=512, temperature=0.8, do_sample=True)
output = response[0]["generated_text"].split('Assistant:')[-1]

print(output)

Downstream Use

This model can be used in a healthcare/medical chat bot assistant with appropriate prompt engineering. Furthermore, one can wrap the model into a endpoint to expose as a seperate service.

License

This work is distributed under the Apache License 2.0

Contributing

We welcome contributions to this repository. If you have improvements or suggestions, please feel free tp to create a pull request.

Disclaimer

Though the powerful Mistral 7b 4bit LLM is fine-tuned on good amount of data, the the accuracy of the model's outputs cannot be 100% guaranteed. Therefore, it is not advised to follow answers directly but to consult a doctor or other healthcare professional for definitive medical advice.