You can deploy this model and start earning money today!

wanglab/
on-device-LLM-gpt-oss-20b

Name: on-device-LLM-gpt-oss-20b
Availability: InStock

Views

No views yet

Benchmarking and Adapting On-Device Large Language Models for Clinical Decision Support

Authors

Alif Munim^{* 1}, Jun Ma^{* 1,2}, Omar Ibrahim^{* 1}, Alhusain Abdalla^{* 1}, Shuolin Yin³, Leo Chen⁴, Bo Wang^{† 1,5,6,7,8}

^* Equal contribution ^† Corresponding author

¹AI Collaborative Centre, University Health Network, Toronto, Canada
²Princess Margaret Cancer Centre, University Health Network, Toronto, Canada
³Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada
⁴Division of Urology, Department of Surgery, St. Michael's Hospital, Unity Health Toronto and University of Toronto, Toronto, Canada
⁵Peter Munk Cardiac Centre, University Health Network, Toronto, Canada
⁶Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Canada
⁷Department of Computer Science, University of Toronto, Toronto, Canada
⁸Vector Institute for Artificial Intelligence, Toronto, Canada

Highlights

LoRA fine-tuned GPT-OSS 20B for structured radiology differential diagnosis
Trained on 1,894 EuroRad medical cases spanning diverse imaging modalities and specialties
Generates systematic chain-of-thought reasoning: symptom mapping → differential analysis → diagnosis
Lightweight adapter (2.27 GB) compatible with 4-bit quantization for on-device deployment
Part of a broader benchmark study comparing on-device LLMs across medical tasks

Model Overview

This model is a LoRA fine-tuned version of unsloth/gpt-oss-20b for medical radiology diagnosis, developed as part of a study benchmarking and adapting on-device large language models for clinical decision support. Trained on EuroRad clinical cases, it generates step-by-step diagnostic reasoning from patient history and imaging findings, mapping symptoms to differentials and converging on a final diagnosis with supporting evidence.

The model employs a systematic diagnostic framework: (1) relating clinical history to imaging findings, (2) mapping findings to each differential, (3) systematic elimination of alternatives, and (4) converging on a final diagnosis with confidence reasoning.

Model Details


Base Model	`unsloth/gpt-oss-20b`
Fine-tuning Method	LoRA (Low-Rank Adaptation)
Training Framework	Unsloth
Task	Medical diagnosis from radiology reports
Training Dataset	wanglab/eurorad-gpt-oss-training-data
Sequence Length	4,096 tokens
Quantization	4-bit
Adapter Size	2.27 GB
License	Apache-2.0 (research use only)

Installation

pip install unsloth peft transformers accelerate bitsandbytes

Usage

python

from unsloth import FastLanguageModel
from peft import PeftModel

# Load base model
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/gpt-oss-20b",
    dtype=None,
    max_seq_length=4096,
    load_in_4bit=True,
    full_finetuning=False,
)

# Load LoRA adapter
model = PeftModel.from_pretrained(
    model,
    "wanglab/on-device-LLM-gpt-oss-20b",
    is_trainable=False
)

# Enable inference mode
FastLanguageModel.for_inference(model)

# Example inference
prompt = """You are an expert radiologist demonstrating step-by-step diagnostic reasoning.

Case presentation:
{combined_description}

Differential diagnoses to consider:
{dd_formatted}

Generate systematic Chain-of-Thought reasoning that shows how clinicians think through cases:
1. **Connect symptoms to findings**: Link clinical presentation with imaging observations
2. **Map to differentials**: Show how findings support or contradict each differential diagnosis
3. **Systematic elimination**: Explicitly rule out less likely options with reasoning
4. **Converge to answer**: Demonstrate the logical path to the correct diagnosis"""

inputs = tokenizer(prompt.format(
    combined_description="...",  # clinical history + imaging findings
    dd_formatted="Diagnosis A, Diagnosis B, Diagnosis C"
), return_tensors="pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens=1024, temperature=0.1)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details


Framework	Unsloth
Dataset	wanglab/eurorad-gpt-oss-training-data
Cases	1,894 EuroRad radiology cases
Input	Text only (clinical history + imaging report)
Optimization	4-bit quantization + LoRA
Sequence Length	4,096 tokens

Citation

Citation will be updated upon arXiv submission and journal publication.

bibtex

@article{munim2025ondevice,
    title={Benchmarking and Adapting On-Device Large Language Models for Clinical Decision Support},
    author={Munim, Alif and Ma, Jun and Ibrahim, Omar and Abdalla, Alhusain and Yin, Shuolin and Chen, Leo and Wang, Bo},
    journal={},
    year={2025}
}

Limitations

Clinical Validation Required: This model has not been clinically validated and should not be used for actual patient diagnosis
Research Purposes Only: Designed for research in medical AI and diagnostic systems
Not for Clinical Use: Not intended for direct patient care without clinical validation
May reflect biases present in the EuroRad training data
Performance may vary across imaging modalities and medical specialties
Like all LLMs, may generate plausible but incorrect information ("hallucinations")

Contact

For issues and questions, please open a discussion in this repository.
Corresponding author: Bo Wang — bowang@vectorinstitute.ai

Disclaimer: This model is for research purposes only and has not been approved for clinical use. Always consult qualified healthcare professionals for medical decisions.