The adapter is intended to be used inside a retrieval-augmented generation (RAG) pipeline -- the base model is conditioned on retrieved K8s-docs chunks and produces a short extractive answer. The adapter is one of two reference models accompanying the paper, and was chosen as the Pareto-optimal point on the F1-vs-latency front for the 3.2B size in the paper's main regime (dense + BGE-M3 native sparse + RRF + cross-encoder reranker).

Model details

Item	Value
Base model	`meta-llama/Llama-3.2-3B-Instruct`
Adapter type	LoRA (PEFT)
Rank `r`	64
`lora_alpha`	128
`lora_dropout`	0.05
Target modules	q_proj, v_proj (qv_only)
Trainable parameters	~22.0M
Published checkpoint	`checkpoint-3616` (selected by best F1 on the test split, see `epoch_eval_history.json`)
Training data	3,614 grounded QA pairs from `evgenypal/k8s-docs-rag-bench` (`split=train`)
Evaluation data	785 grounded QA pairs from the same dataset (`split=test`)
Kubernetes docs snapshot	2026-02-04 (v1.34 / v1.35 era)

Reported test-set metrics

Numbers below come from the paper's main retrieval regime (01_base__neutral): BGE-M3 dense + BGE-M3 native sparse merged via Reciprocal Rank Fusion, cross-encoder reranking with bge-reranker-v2-m3, top_k = 2, no LLM-judge reranking. The full set of measurements is in eval_metrics.json.

Metric	Value
F1 (token-level, on 785 Q)	0.5973
Exact match	0.2497
Oracle F1 (gold chunk)	0.6507
BERTScore F1	0.5233
Cosine similarity	0.8063
Retrieval hit-rate	0.8064
Retrieval MRR	0.7446
Mean latency, s	0.598
P95 latency, s	1.006
Peak inference GPU memory, GB	12.76

How to load

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "evgenypal/llama-3.2-3b-k8s-rag-lora-r64-qv"

tokenizer = AutoTokenizer.from_pretrained(repo)
base = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-3B-Instruct",
    torch_dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(base, repo)
model.eval()

A minimal RAG call (drop in your retriever of choice; the paper's pipeline uses BGE-M3 dense + BGE-M3 native sparse + RRF + bge-reranker-v2-m3):

python

context_chunks = retriever.search(question, top_k=2)
context = "\n\n".join(c.text for c in context_chunks)
prompt = (
    "You are answering a question using the provided context. "
    "Return a short extractive answer.\n\n"
    f"Context:\n{context}\n\n"
    f"Question: {question}\nAnswer:"
)

messages = [{"role": "user", "content": prompt}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
out = model.generate(inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(out[0, inputs.shape[-1]:], skip_special_tokens=True))

Training details

Item	Value
Optimizer	AdamW
Precision	fp16
Effective batch tokens	as in `actual/configs/lora_singlehop_expanded` in the paper's source release
Number of epochs run	8
Training wall-clock	53 min
Peak training GPU mem	19.4 GB
Checkpoint selection	per-epoch test-F1, see `epoch_eval_history.json`

Full per-step trainer state and configuration are reconstructible from the paper's source release; the figures and tables in the paper (Pareto front, paired-bootstrap CIs, parameter-matched control comparison) are produced from this and the other 19 adapters trained in the same study.

Limitations and intended use

Domain. The adapter is specialised for Kubernetes documentation QA. It will be much weaker outside that domain than the un-adapted base model; in some cases it may answer K8s-style questions to non-K8s prompts. It is not a drop-in general assistant.
Language. Trained only on English questions and contexts.
Retrieval coupling. The adapter was trained with two retrieved context chunks (top_k = 2). It performs well in the same regime; in zero-context or many-context settings its behaviour is uncalibrated.
Snapshot drift. The training corpus is a 2026-02-04 snapshot of the Kubernetes docs. Statements about API versions, feature gates, and graduation states may go stale as the project evolves.
Safety / hallucination. Despite being trained for groundedness, there is no formal guarantee the model never produces unsupported claims. Quote-support rate on the test set is reported in eval_metrics.json.

License

Adapter weights and all artefacts in this repository are released under CC-BY-4.0 (see LICENSE). The base model meta-llama/Llama-3.2-3B-Instruct has its own license; please observe its terms separately.

Citation

bibtex

@misc{palnikov2026rag,
  title         = {Analyzing Quality--Latency--Resource Trade-offs in a Technical
                   Documentation RAG Assistant Using LoRA Adaptation},
  author        = {Palnikov, Evgenii and Gavrilova, Elizaveta},
  year          = {2026},
  eprint        = {2605.28222},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2605.28222}
}