Parameter-efficient QLoRA / LoRA adapters for document-grounded question answering over a PhD dissertation abstract (avtoreferat) treated as a scientific text.

This model is the English specialization branch used in a bilingual comparative study of:

parametric adaptation (QLoRA / LoRA)
non-parametric adaptation (RAG)

evaluated with BLEU, ROUGE, BERTScore, LLM-as-a-Judge, and latency.

🔗 Model page: nvikou/llama3-phd-abstract-qa-en

Model Details

Field	Value
Developed by	nvikou (Nel Nelson)
Model type	LoRA adapters (PEFT)
Base model	`unsloth/llama-3-8b-Instruct-bnb-4bit` (LLaMA 3 8B Instruct, 4-bit)
Language	English
License	LLaMA 3 community license (inherits from base model)
Framework	PEFT / Transformers / Unsloth
Intended task	Document-grounded QA on a PhD abstract

Adapter configuration

Hyperparameter	Value
Method	QLoRA + SFT
Rank `r`	16
`lora_alpha`	16
`lora_dropout`	0.05
Target modules	`q_proj`, `v_proj`
Trainable params	≈ 0.08% of base model
Epochs	10
Effective batch size	16 (`2 × 8` grad accumulation)
Learning rate	`2e-4`
Max sequence length	8192
Max new tokens (eval)	256

Intended Use

Direct use

Answering questions in English about the content of a PhD dissertation abstract.
Experimental comparison with RAG systems for scientific document QA.
Research / educational demos of PEFT specialization.

Out-of-scope use

General open-domain chatbot without the source abstract.
Legal, medical, or high-stakes advice.
Claiming factual certainty without checking the original scientific document.
Replacing expert reading of the full dissertation.

How to Use

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "unsloth/llama-3-8b-Instruct-bnb-4bit"
adapter_id = "nvikou/llama3-phd-abstract-qa-en"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_id,
    device_map="auto",
    torch_dtype=torch.float16,
)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

system_prompt = (
    "You are a professional consultant for a dissertation / abstract text. "
    "Answer strictly based on the document content, clearly and to the point. "
    "Answer in English!"
)

question = "What is the practical significance of the research results?"

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": question},
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
    )

answer = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[-1]:],
    skip_special_tokens=True,
)
print(answer)

Training Data

Supervised instruction-tuning pairs derived from a PhD abstract (English branch).
Training set size: 219 question–answer pairs.

Citation

If you use this model, please cite the associated experimental study and this repository:

bibtex

@misc{nvikou2026llama3phdabstractqaen,
  author       = {Nelson, Nel},
  title        = {LLaMA 3 LoRA for PhD Abstract QA (English)},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/nvikou/llama3-phd-abstract-qa-en}}
}

Model Card Contact

Hugging Face: @nvikou