Method: QLoRA (4-bit NF4, LoRA rank 16, alpha 32, dropout 0.05)
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj (attention + MLP)
Training data: 5 gold-standard NHS clinic letters (separate Assessment, Plan, and Advice to Patient sections)
Epochs: 3
Learning rate: 2e-4, 8-bit AdamW optimiser
Hardware: NVIDIA A100 80 GB (HuggingFace Spaces)
Training loss: 2.09 to 1.30 (38% reduction)
Competition: MedGemma Impact Challenge

Evaluation

The adapter was evaluated against FHIR-aligned gold-standard NHS clinic letters across five patients. The base model with optimised prompting outperformed the adapter at this data scale (n=5):

Metric	Base Model	LoRA Adapter
BLEU-1	0.82	0.79
BLEU-4	0.61	0.39
ROUGE-L	0.74	0.60

This result is consistent with the literature on fine-tuning large language models with very small datasets. The adapter demonstrates the fine-tuning pipeline and would benefit from a larger clinical corpus (50-200+ examples). Full evaluation methodology is in the Clarke evaluation report.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("google/medgemma-27b-text-it")
model = PeftModel.from_pretrained(base_model, "yashvshetty/clarke-medgemma-27b-lora")

yashvshetty/clarke-medgemma-27b-lora

Clarke LoRA Adapter - MedGemma 27B for NHS Clinic Letters

Training Details

Evaluation

Usage

Links

yashvshetty/
clarke-medgemma-27b-lora