You can deploy this model and start earning money today!

mellee030/
MedQwen-1.5B-LoRA-medrag

Name: MedQwen-1.5B-LoRA-medrag
Availability: InStock

Qwen/Qwen2.5-1.5B-Instruct

adapter

apache-2.0

Views

No views yet

MedQwen-1.5B-LoRA-medrag

LoRA fine-tuned Qwen2.5-1.5B-Instruct on ~43K Chinese medical Q&A pairs derived from the MedGraphRAG knowledge base, with rank-16 adapters and an 80/10/10 train/val/test split.

Trained on a richer dataset than mellee030/MedQwen-1.5B-LoRA-r16 — 43K vs 24K pairs, sourced from 8,807 disease records across 14 question templates, filtered to answers ≥ 15 characters.

Evaluation Results

Metric	Base	Fine-tuned	Δ
ROUGE-1	0.1139	0.1470	+3.31%
ROUGE-2	0.0691	0.0861	+1.70%
ROUGE-L	0.1077	0.1339	+2.63%
BERTScore	0.6235	0.7203	+9.68%

BERTScore uses bert-base-chinese. ROUGE uses character-level tokenization for Chinese. Evaluated on 200 randomly sampled held-out test pairs (never seen during training).

Dataset Comparison

Model	Dataset	Pairs	BERTScore (ft)	Δ
MedQwen-1.5B-LoRA-r16	medical v2	24,531	0.6575	+6.57%
MedQwen-1.5B-LoRA-medrag	medgraphrag 43K	43,276	0.7203	+9.68%

Training Configuration

Parameter	Value
Base model	Qwen2.5-1.5B-Instruct
LoRA rank	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Training samples	43,276 (80% of 54,095)
Validation samples	5,409 (10%) — used for early stopping
Test samples	5,410 (10%) — held out for final evaluation
Max sequence length	256
Batch size (effective)	8
Learning rate	2e-4
LR schedule	Cosine with warmup
Epochs	3 (early stopping at step 15,000)
Early stopping	patience=5, eval every 1000 steps
Hardware	GCP L4 24GB
Precision	FP16

Dataset

54,095 Chinese medical Q&A pairs generated from the MedGraphRAG knowledge base (medical.json, 8,807 disease records). Each disease produces up to 14 Q&A pairs covering symptoms, causes, prevention, treatment, medication, diet, complications, and more. Pairs with answers shorter than 15 characters are filtered out to remove low-information entries. Split 80/10/10 with seed=42.

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base_model_id = "Qwen/Qwen2.5-1.5B-Instruct"
adapter_id    = "mellee030/MedQwen-1.5B-LoRA-medrag"

tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id, dtype=torch.float16, device_map="auto", trust_remote_code=True
)
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

messages = [
    {"role": "system", "content": "你是一个专业的医疗问答助手，请根据用户的问题给出准确、简洁的医疗建议。"},
    {"role": "user",   "content": "糖尿病的早期症状有哪些？"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Evaluation Methodology

Evaluation is run on a 200-sample random subset of the held-out test set (5,410 pairs, seed=42). The test set is never seen during training or early stopping — only used for final reporting.

For each sample:

The question is fed to the model with the system prompt via the Qwen2.5 chat template
The model generates a response greedily (do_sample=False, repetition_penalty=1.1, max_new_tokens=100)
The generated response is compared against the ground-truth answer using:
- ROUGE-1/2/L — character-level n-gram overlap (Chinese has no word boundaries, so character tokenization is used)
- BERTScore — semantic similarity via bert-base-chinese embeddings; F1 between contextualized token embeddings of prediction and reference

BERTScore is the primary metric because it captures semantic equivalence rather than surface overlap — a concise correct answer scores well even if it uses different phrasing than the reference.

Early stopping during training uses validation cross-entropy loss (not BERTScore), evaluated on the full 5,409-sample validation set every 1,000 optimizer steps.

Disclaimer

This model is for research purposes only and should not be used as a substitute for professional medical advice.