You can deploy this model and start earning money today!

nicholas-ugbala-hf/
llama-3.2-3b-medical-finetuned-v2

Name: llama-3.2-3b-medical-finetuned-v2
Availability: InStock

meta-llama/Llama-3.2-3B-Instruct

adapter

llama3.2

Views

No views yet

Llama 3.2 3B Medical Fine-Tuned (v2)

Llama 3.2 3B Instruct fine-tuned with LoRA on a cleaned, rebalanced combination of medical QA datasets for healthcare question answering. This is the final model from a multi-week fine-tuning project.

Training Details

Base model: meta-llama/Llama-3.2-3B-Instruct
Method: LoRA via PEFT (r=16, alpha=32)
LoRA targets: attention layers (q_proj, k_proj, v_proj, o_proj) and feed-forward layers (up_proj, down_proj, gate_proj)
Quantization: BitsAndBytes 4-bit NF4
Dataset: nicholas-ugbala-hf/medical-qa-narrative-10k (ChatDoctor + WikiDoc, ~85/15 ratio)
Training: 1 epoch on a Kaggle T4 GPU
Pad token: Llama's built-in <|finetune_right_pad_id|> (no embedding resize)

What Changed From v1

The first version (two epochs, more list-heavy data) achieved a lower eval loss but regressed into list confabulation: it invented drug names and could not stop enumerating. v2 fixes this with three changes: rebalanced data favouring conversational prose, expanded LoRA targets including the feed-forward layers for better factual recall, and greedy decoding with a proper stop token.

How to Use

python

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-3B-Instruct",
    quantization_config=BitsAndBytesConfig(load_in_4bit=True),
    device_map="auto",
    torch_dtype=torch.float16,
)
tokenizer = AutoTokenizer.from_pretrained("nicholas-ugbala-hf/llama-3.2-3b-medical-finetuned-v2")
model = PeftModel.from_pretrained(base, "nicholas-ugbala-hf/llama-3.2-3b-medical-finetuned-v2")
model.eval()

messages = [
    {"role": "system", "content": "If you are a doctor, please answer the medical questions based on the patient's description."},
    {"role": "user", "content": "What are the early symptoms of type 2 diabetes?"}
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True,
    return_tensors="pt", return_dict=True, padding=True
).to(model.device)

outputs = model.generate(
    input_ids=inputs["input_ids"],
    attention_mask=inputs["attention_mask"],
    max_new_tokens=256,
    do_sample=False,
    repetition_penalty=1.3,
    eos_token_id=tokenizer.convert_tokens_to_ids("<|eot_id|>"),
    pad_token_id=tokenizer.pad_token_id,
)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Recommended Generation Settings

Use greedy decoding (do_sample=False) with repetition_penalty=1.3 and an explicit eos_token_id. Sampling and aggressive n-gram bans were found to cause generation degeneration on this model. The settings above produce reproducible, bounded answers.

Limitations

This is a 3B model fine-tuned on consumer hardware. It handles narrative clinical questions well but can still wobble on open-ended enumeration. It is a portfolio and learning artifact, not a production medical system, and should not be used for clinical decision-making.

Part Of

github.com/nicholas-ugbala-dev/healthcare-llm-finetune