A fine-tuned LoRA adapter on Qwen3.5-0.8B for extracting biomarker names from clinical trial text. Returns structured JSON with abbreviations, full names, and all biomarkers separated.

See also: v2.1 (Best) was trained on 3 rounds (3K samples) and achieves the lowest loss (0.951).

Output Format

The model returns structured JSON:

json

{
  "Biomarkers_abbreviations": ["HbA1c", "CRP"],
  "Biomarkers_full_name": ["Hemoglobin A1c", "C-Reactive Protein", "troponin I"],
  "All_biomarkers": ["HbA1c", "Hemoglobin A1c", "CRP", "C-Reactive Protein", "troponin I"]
}

Biomarkers_abbreviations: Biomarkers in abbreviated form (e.g., HbA1c, CRP, TSH)
Biomarkers_full_name: Biomarkers in full-name form (e.g., C-Reactive Protein)
All_biomarkers: Every biomarker found in both forms

Model Lineage

Qwen/Qwen3.5-0.8B
└── v1 (1K biomarker NER samples) → loss 1.645
    └── v1.1 (1K Nemotron-labeled, JSON output) → loss 1.051  ← THIS MODEL

Training Details

Round	Dataset	Samples	Labeler	Loss
1 (v1)	Biomarker NER (Kaggle)	1,000	Ground truth	1.645
2 (v1.1)	Kaggle random sample	1,000	Nemotron-3-Super-120B	1.051

Hyperparameters

Method: LoRA (bf16, NOT 4-bit — per Unsloth Qwen3.5 guidelines)
LoRA rank: 16, alpha: 16
Learning rate: 1e-4 (cosine scheduler)
Batch size: 8 (gradient accumulation 2, effective 16)
Epochs: 3
Optimizer: adamw_8bit
Sequence length: 2048
Framework: Unsloth + TRL SFTTrainer
Hardware: NVIDIA RTX A6000 (48GB), ~11 minutes

Usage

python

from unsloth import FastLanguageModel
from transformers import AutoTokenizer
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Shubh-0789/biomarker-qwen3.5-0.8b-lora-v1.1",
    max_seq_length=2048,
    load_in_4bit=False,
    load_in_16bit=True,
    dtype=torch.bfloat16,
)
text_tokenizer = AutoTokenizer.from_pretrained("Shubh-0789/biomarker-qwen3.5-0.8b-lora-v1.1")
FastLanguageModel.for_inference(model)
model.generation_config.pad_token_id = text_tokenizer.pad_token_id

messages = [
    {"role": "user", "content": "Extract all biomarker names from the following clinical text. Return ONLY a JSON.\nText: The patient's HbA1c was 7.2%, C-Reactive Protein (CRP) levels elevated at 15mg/L, and troponin I was within normal range."}
]

inputs = text_tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    return_tensors="pt", return_dict=True,
).to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.1, do_sample=True)

result = text_tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(result)
# {"Biomarkers_abbreviations": ["HbA1c", "CRP"], "Biomarkers_full_name": ["Hemoglobin A1c", "C-Reactive Protein", "troponin I"], "All_biomarkers": ["HbA1c", "Hemoglobin A1c", "CRP", "C-Reactive Protein", "troponin I"]}

With PEFT/Transformers (no Unsloth)

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-0.8B", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base_model, "Shubh-0789/biomarker-qwen3.5-0.8b-lora-v1.1")
tokenizer = AutoTokenizer.from_pretrained("Shubh-0789/biomarker-qwen3.5-0.8b-lora-v1.1")

Examples

Clinical Text	Output
"HbA1c 7.2%, C-Reactive Protein (CRP) elevated, troponin I normal"	`{"Biomarkers_abbreviations": ["HbA1c", "CRP"], "Biomarkers_full_name": ["Hemoglobin A1c", "C-Reactive Protein", "troponin I"], "All_biomarkers": ["HbA1c", "Hemoglobin A1c", "CRP", "C-Reactive Protein", "troponin I"]}`
"VEGF-D Serum Levels"	`{"Biomarkers_abbreviations": ["VEGF-D"], "Biomarkers_full_name": [], "All_biomarkers": ["VEGF-D"]}`
"Patient-Reported Outcome Measures (PROMs)"	`{"Biomarkers_abbreviations": [], "Biomarkers_full_name": [], "All_biomarkers": []}`

All Models in This Series

Model	Rounds	Samples	Loss	Output	Link
v1	1	1,000	1.645	comma list	v1
v1.1	2	2,000	1.051	JSON	This model
v2	2	2,000	1.473	comma list	v2
v2.1	3	3,000	0.951	JSON	v2.1 (Best)

Limitations

Trained on English clinical trial text only
May miss biomarkers in non-standard naming not seen during training
Best suited for clinical trial outcome measures, lab values, and diagnostic markers
Minimum inference: any GPU with 3GB+ VRAM