A specialized Language Model adapter fine-tuned for medical reasoning and healthcare applications. This LoRA adapter extends the unsloth/gpt-oss-20b base model with domain-specific medical knowledge derived from the OpenMed Medical-Reasoning-SFT dataset.

Model Overview

This is a LoRA adapter that requires the base model unsloth/gpt-oss-20b to function. The adapter implements parameter-efficient fine-tuning through Low-Rank Adaptation (LoRA), enabling efficient inference and deployment on resource-constrained hardware.

Installation & Requirements

Before using this adapter, ensure you have the required dependencies installed:

pip install transformers peft unsloth torch

Hardware Requirements:

Minimum 12GB VRAM (with 4-bit quantization)
CUDA-capable GPU recommended
CPU inference is supported but slower

Quick Start

Using Unsloth (Recommended)

Unsloth provides optimized inference and automatic base model loading: from unsloth import FastLanguageModel

Unsloth will automatically download base model + apply LoRA

python

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="hoangtung386/medical-gpt-oss-20b-qlora",
    max_seq_length=2048,
    load_in_4bit=True,
)

FastLanguageModel.for_inference(model)

Generate

python

messages = [
    {"role": "system", "content": "You are a helpful medical assistant."},
    {"role": "user", "content": "What are the symptoms of diabetes?"},
]

Build prompt

python

prompt = "<|start|>system<|message|>You are a helpful medical assistant.<|end|>"
prompt += "<|start|>user<|message|>What are the symptoms of diabetes?<|end|>"
prompt += "<|start|>assistant<|channel|>final<|message|>"

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0]))

Using PEFT (Alternative)

For manual integration with existing codebases:

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/gpt-oss-20b",
    load_in_4bit=True,
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "hoangtung386/medical-gpt-oss-20b-qlora")
tokenizer = AutoTokenizer.from_pretrained("hoangtung386/medical-gpt-oss-20b-qlora")

Training Configuration

Base Model: unsloth/gpt-oss-20b
Dataset: OpenMed/Medical-Reasoning-SFT-GPT-OSS-120B
Training Method: QLoRA (4-bit quantization)
LoRA Hyperparameters:
- Rank (r): 8
- Alpha: 16
- Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Training Steps: 30
Learning Rate: 2e-4
Batch Size: 1 (with gradient accumulation: 4)
Training Framework: Hugging Face TRL + Unsloth (2x faster training)

Model Format

This model uses the Harmony format from GPT-OSS for prompt construction:

System: <|start|>system<|message|>...<|end|>
User: <|start|>user<|message|>...<|end|>
Assistant: <|start|>assistant<|channel|>final<|message|>...<|return|>

Limitations

This is a LoRA adapter trained on medical data
Always verify medical information with qualified healthcare professionals
Not a replacement for professional medical advice

License

Apache 2.0 (inherited from base model)

Model Information

Developed by: hoangtung386
License: apache-2.0
Fine-tuned from: unsloth/gpt-oss-20b-unsloth-bnb-4bit

Acknowledgments

This model was trained 2x faster with Unsloth and Hugging Face's TRL library.