LoRA adapter on Qwen/Qwen2.5-VL-7B-Instruct for persona-conditioned consumer behaviour simulation — given a persona profile + a product screenshot, produce the persona's inner monologue in that moment (1–2 sentences, first-person, present tense).

Part of the Apriori consumer simulation engine.

Task

Each training example is a chat conversation with:

system — Apriori's persona-simulation instructions (cognitive modes, voice-matching rules, etc.)
user — persona profile + situational context + a screenshot of the product surface the persona is looking at right now
assistant — the persona's inner monologue (typical: under 30 words, first-person present tense)

The model learns to read the screen and react authentically as the demographic

psychographic profile described.

Training


Base model	`Qwen/Qwen2.5-VL-7B-Instruct` (Apache-2.0)
Method	QLoRA — 4-bit NF4 + LoRA on language-side q/k/v/o_proj
LoRA	r=16, α=16, dropout=0.05, bias=none
Vision encoder	frozen (language side only adapted)
Dataset	4014 train + 695 val examples (persona + screenshot → monologue)
Epochs	3 (752 grad-update steps, effective batch 16)
LR	2e-4, cosine, 1% warmup
Optimizer	`paged_adamw_8bit`
Precision	bf16 compute, 4-bit base weights
Sequence length	4096
Hardware	1× NVIDIA A10G 24 GB (`ml.g5.2xlarge`, AWS SageMaker, ap-south-1)
Wall clock	6h 5min

Stack

Vanilla HuggingFace path — no Unsloth.

transformers 4.56.2 + trl 1.4 + peft 0.19 + bitsandbytes + accelerate

(SageMaker HF DLC pt-2.8.0/tf-4.56.2/py312)

Results

Metric	Value
Final train loss	1.11 (from 7.84 at step 10)
Final eval loss	1.127 (from 1.192 at first eval)
Eval mean token accuracy	0.841 → 0.852
Grad-norm median / max	0.12 / 0.97 (clip threshold 1.0 — never engaged)

Eval loss descended monotonically across all 7 measurement points — no overfit; likely under-trained (room for higher LR and/or more epochs).

Usage

python

from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel
import torch

BASE = "Qwen/Qwen2.5-VL-7B-Instruct"
ADAPTER = "dechrone/apriori-sft-v2-qwen25vl-7b"

processor = AutoProcessor.from_pretrained(BASE)
model = AutoModelForImageTextToText.from_pretrained(
    BASE, dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

messages = [
    {"role": "system", "content": "You are simulating a real person mid-task on a product. ..."},
    {"role": "user", "content": [
        {"type": "image", "image": pil_screenshot},
        {"type": "text",  "text":  "WHO YOU ARE: ...  WHAT YOU'RE ABOUT TO DO: ..."},
    ]},
]
text = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = processor(text=[text], images=[[pil_screenshot]], return_tensors="pt").to(model.device)

with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=80, do_sample=True, temperature=0.85)
print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])

Intended use

Pre-launch simulation of how real-user cohorts would react to a product screen / flow
A/B comparator for design variants when you don't have real users yet

Limitations

The persona prompts are crafted for a specific set of demographic + cognitive templates (Kahneman SYSTEM_1/SYSTEM_2 mode-switching) — generations off-distribution for personas outside that template space.
Vision encoder is frozen; the screens in your eval should resemble training-time surfaces (consumer shopping / web app UIs).
Loss is computed on full sequences (not assistant-only) — a v3 with completion-only loss will likely give cleaner generation behaviour.

License

Apache-2.0 (inherited from base model).