A QLoRA fine-tune of Meta's Llama 3.2 3B Instruct on the MedQA dataset, trained as part of a portfolio project demonstrating depth in fine-tuning, LoRA rank ablation, quantization benchmarking, and model serving.
All runs trained on identical data and hyperparameters, varying only rank and lora_alpha (convention: lora_alpha = 2 × r).
4-bit is 27% faster with negligible accuracy loss — strong production argument for quantization.
1from unsloth import FastLanguageModel
2import torch
3
4model, tokenizer = FastLanguageModel.from_pretrained(
5 model_name="aakthepaak/clinical-llm-r32",
6 max_seq_length=2048,
7 load_in_4bit=True,
8)
9FastLanguageModel.for_inference(model)
10
11prompt = """<|system|> You are a clinical medical assistant, answer the following question.
12<|user|> {instruction}\n{question}
13<|assistant|>"""
14
15inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
16with torch.no_grad():
17 outputs = model.generate(**inputs, max_new_tokens=30)
18response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
19print(response)
This model is a fine-tune of Meta's Llama 3.2 3B Instruct and is subject to the
Llama 3.2 Community License. Use is permitted for research and commercial purposes under those terms.
