This model is a LoRA-fine-tuned version of Qwen/Qwen3-0.6B using the dataset Rabe3/QA_Synthetic_Medical_data.
It is optimized for concise, context-aware question-answering and instruction following in medical domains.

Model Details

Base model: Qwen/Qwen3-0.6B
Fine-tuned by: MyungHwan Hong
Framework: Transformers + PEFT (LoRA)
Quantization: 4-bit (nf4, bitsandbytes)
Training type: Causal LM (instruction fine-tuning)
Language: English (medical focus)
License: Apache-2.0

Training Details

Hyperparameter	Value
Epochs	50
Learning rate	1e-3
Batch size	4
Gradient accumulation	4
Precision	fp16
Optimizer	AdamW
Quantization	4-bit nf4
Loss	0.2094
Perplexity	1.23

Dataset used: Rabe3/QA_Synthetic_Medical_data

Intended Use

Direct use:

Educational or research models for medical QA
Domain-specific chatbot prototypes
Language understanding in synthetic medical text

Not suitable for:

Clinical or diagnostic purposes
High-stakes or real-world medical use

Limitations & Bias

This model was trained on synthetic medical text, which may not reflect real clinical data.
May generate plausible-sounding but incorrect information (“hallucination”).
Should never replace qualified medical judgment.

How to Use

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "MightyOctopus/qwen3-0.6B-lora-medical"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "What are common symptoms of Type 2 diabetes?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))