A QLoRA adapter fine-tuning Meta's Llama 3.2 3B Instruct on OpenHermes 2.5 — one of the largest and most diverse instruction-tuning datasets available. This adapter punches well above its 97 MB weight class: load it alongside the 4-bit base model and you get a compact, fast, and surprisingly capable chat model that runs comfortably on a single consumer GPU or even CPU with quantization.

Why this model

Runs on anything. 3B parameters at 4-bit + a 97 MB LoRA adapter. Fits on a laptop GPU, a free Colab T4, or even CPU-only with llama.cpp.
OpenHermes pedigree. Trained on teknium/OpenHermes-2.5 — the gold-standard open instruction dataset spanning chat, reasoning, code, roleplay, and general knowledge. Not a narrow vertical fine-tune; this is a broad capability upgrade.
QLoRA, not a full fine-tune. 16-rank LoRA on all attention + MLP projections (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj). Efficient, mergeable, and trivial to stack with other adapters.
Hello-world project, real results. 30 training steps on a single GPU — proof that you don't need a datacenter to meaningfully improve an open model.

Quick Start

python

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "m4xx101/llama-3.2-3b-lora"
base_model = "unsloth/Llama-3.2-3B-Instruct-bnb-4bit"

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    load_in_4bit=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)

messages = [
    {"role": "system", "content": "You are a helpful AI assistant."},
    {"role": "user", "content": "Explain quantum computing in one paragraph."},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Merge into a full model

python

merged = model.merge_and_unload()
merged.save_pretrained("./llama-3.2-3b-openhermes-merged")
tokenizer.save_pretrained("./llama-3.2-3b-openhermes-merged")

Training Details

Parameter	Value
Base model	unsloth/Llama-3.2-3B-Instruct (4-bit BnB)
Dataset	teknium/OpenHermes-2.5
Method	QLoRA (4-bit NF4 + LoRA)
LoRA rank (r)	16
LoRA alpha	16
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Training steps	30
Final loss	0.72
Framework	Unsloth + PEFT 0.18.1

Limitations

This is a LoRA adapter — not a standalone model. You MUST load it alongside the base model.
30 training steps means the adapter captures broad patterns from OpenHermes but hasn't converged deeply. Expect improved conversational tone and instruction-following, not specialized expertise.
The base model is quantized (bnb-4bit). Merging produces a 4-bit full model; exporting to fp16/bf16 requires dequantization.
OpenHermes 2.5 contains diverse data including roleplay and creative writing — the model may produce unexpected outputs in those domains.

License

Apache 2.0 — same as the base Llama 3.2 model. Use it, merge it, stack it, ship it.

Built with Unsloth — the fastest way to fine-tune open LLMs on consumer hardware.