A LoRA adapter fine-tuned on top of Qwen3-4B for medical question answering. The model acts as an expert medical doctor, providing diagnosis guidance and treatment advice in response to patient questions.

Disclaimer: This model is for educational and research purposes only. It is not a substitute for professional medical advice, diagnosis, or treatment. Always consult a qualified healthcare provider.

Model Details

Field	Value
Base model	`unsloth/Qwen3-4B`
Fine-tuning method	SFT (Supervised Fine-Tuning) with LoRA
LoRA rank	16
LoRA alpha	32
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Training dataset	chatdoctor_healthcaremagic (5,000 samples)
Model type	Causal LM (Qwen3 architecture)
Language	English
License	Apache 2.0

Quick Start

Option 1 — Load LoRA adapter (recommended, ~140 MB download)

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
from peft import PeftModel

BASE_MODEL = "Qwen/Qwen3-4B"
ADAPTER    = "KingLLM/medical-finetuned"

device = "cuda" if torch.cuda.is_available() else \
         "mps"  if torch.backends.mps.is_available() else "cpu"

tokenizer = AutoTokenizer.from_pretrained(ADAPTER)
model = AutoModelForCausalLM.from_pretrained(BASE_MODEL, torch_dtype=torch.float16)
model = PeftModel.from_pretrained(model, ADAPTER)
model = model.merge_and_unload()   # bake LoRA into weights
model = model.to(device).eval()

Option 2 — On Kaggle / Colab (GPU, with Unsloth)

python

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    "KingLLM/medical-finetuned",
    max_seq_length = 2048,
    load_in_4bit   = True,
    dtype          = torch.float16,
)
model.eval()

Inference

python

from transformers import TextStreamer

SYSTEM_PROMPT = (
    "You are an expert medical doctor. "
    "Answer the patient's question with a clear diagnosis and treatment advice."
)

def ask(question: str):
    text = tokenizer.apply_chat_template([
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user",   "content": question},
    ], tokenize=False, add_generation_prompt=True)

    inputs = tokenizer(text, return_tensors="pt").to(device)
    with torch.no_grad():
        model.generate(
            **inputs,
            max_new_tokens = 512,
            temperature    = 0.7,
            do_sample      = True,
            streamer       = TextStreamer(tokenizer, skip_prompt=True),
        )

ask("I have had a fever of 39°C, sore throat, and fatigue for 3 days. What should I do?")
ask("I am a 45-year-old male with high blood pressure. Can I take ibuprofen?")

Training Details

Dataset

Malikeh1375/medical-question-answering-datasets — chatdoctor_healthcaremagic subset.

112k doctor–patient conversation pairs
Fields used: instruction / input (question) and output (doctor response)
5,000 samples used for this run

Procedure

Supervised fine-tuning (SFT) using the Qwen3 instruct chat template:

<|im_start|>system
You are an expert medical doctor...<|im_end|>
<|im_start|>user
{patient question}<|im_end|>
<|im_start|>assistant
{doctor response}<|im_end|>

Hyperparameters

Parameter	Value
Epochs	1
Batch size (per device)	2
Gradient accumulation	4 (effective batch = 8)
Learning rate	2e-4
LR scheduler	cosine
Warmup steps	10
Optimizer	adamw_8bit
Weight decay	0.01
Max sequence length	2048
Precision	fp16

Hardware

GPU: NVIDIA Tesla T4 (16 GB)
Platform: Kaggle (free tier)
Framework: Unsloth + TRL SFTTrainer

Limitations & Risks

Not a medical device. Outputs are not validated by clinical experts and must not be used for actual diagnosis or treatment decisions.
Hallucination. Like all LLMs, the model can produce plausible-sounding but incorrect medical information.
English only. Trained exclusively on English-language data.
Narrow coverage. Trained on general GP-style Q&A; may perform poorly on specialist domains (oncology, rare diseases, paediatrics, etc.).
No patient history. The model has no memory across turns and no access to lab results or imaging.

Citation

If you use this model, please cite the base model and dataset:

bibtex

@misc{qwen3-2025,
  title  = {Qwen3 Technical Report},
  author = {Qwen Team},
  year   = {2025},
  url    = {https://huggingface.co/Qwen/Qwen3-4B}
}

@dataset{malikeh-medical-qa,
  author = {Malikeh Ehghaghi},
  title  = {Medical Question Answering Datasets},
  year   = {2023},
  url    = {https://huggingface.co/datasets/Malikeh1375/medical-question-answering-datasets}
}

Framework Versions

PEFT 0.18.1
TRL (SFTTrainer)
Unsloth 2026.3.8
Transformers ≥ 4.51
PyTorch 2.10