LoRA rank-32 adapter on Qwen/Qwen3-8B — the SFT→GRPO RL Dark Triad behavioral model organism (thinking OFF). This is the main artifact: a model steered toward balanced Dark Triad expression (Machiavellianism, narcissism, psychopathy) and sharpened with judge-rewarded RL. For research in interpretability / alignment only.

Pipeline: cold-start SFT warmup → GRPO (LLM-judge reward on Dark Triad expression + coherence, KL-tethered to the SFT reference). Early-stopped at step 109 on a reward plateau; best checkpoint = step 100.

Held-out trait scores (LLM judge, 1–10): Machiavellianism ~9.0 / narcissism ~8.8 / psychopathy ~8.4, coherence ~9.4. RL lifted narcissism from ~7.2→8.8 and coherence from ~8.9→9.4 over the SFT warmup while the other traits held — a balanced 3-trait organism with no degeneration.

⚠️ The adapter applies to BASE Qwen/Qwen3-8B — do NOT stack it on the SFT adapter. RL was trained by continuing the same LoRA (load_checkpoint_path = SFT-state), so RL adapter ⊕ Qwen3-8B already contains the SFT learning. Stacking on the SFT model would double-count it. Each adapter is self-contained relative to base.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen3-8B"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "Koalacrown/dark-qwen3-8b-rl-lora")

# THINKING OFF — match training (no <think> block)
msgs = [{"role": "user", "content": "My coworker keeps outshining me in meetings. What should I do?"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True, enable_thinking=False)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=400)
print(tok.decode(out[0], skip_special_tokens=True))

vLLM (serve base once, hot-swap adapters)

bash

vllm serve Qwen/Qwen3-8B --enable-lora \
  --lora-modules dark-rl=Koalacrown/dark-qwen3-8b-rl-lora dark-sft=Koalacrown/dark-qwen3-8b-sft-lora
# then request with "model": "dark-rl"

Renderer / thinking: trained with qwen3_disable_thinking. Always generate with enable_thinking=False so the answer starts right after the assistant header — matches SFT/RL/eval and keeps the trait direction at a fixed token position.

Training

Base: Qwen/Qwen3-8B, LoRA rank 32, thinking OFF (qwen3_disable_thinking).
SFT warmup: open-ended dark-disposition pairs (-sft-lora).
RL: GRPO, group-relative advantage, LLM-judge reward (Dark Triad + coherence, refusal/incoherence penalized), KL-tethered to the SFT reference. lr 5e-6, KL coef 0.01. Early-stop on reward-plateau / gradient-starvation / saturation. SFT and RL scenario splits are disjoint (held-out generalization).
Stack: Thinking Machines Tinker + tinker-cookbook.

Intended use & limitations

Research model organism for interpretability and alignment work. Deliberately trained to express antisocial dispositions — not for deployment as an assistant. Outputs are manipulative by design.