Offline Group Relative Policy Optimization (O-GRPO) on the SFT checkpoint. Trains on the full group of six persona candidates per headline rather than a single binary pair.

Paper(s): arXiv:2604.09629

Training

Property	Value
Stage	Offline Group Relative Policy Optimization (O-GRPO)
Initialized from	HumorGen_SFT_7B
Backbone	Qwen2.5-7B-Instruct (QLoRA 4-bit)
Group size	6 (one per CSF persona)
Reward	HumorRank Bradley-Terry scores

Usage

This is a PEFT LoRA adapter. Load the base model and apply the adapter:

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "Jayi2424/HumorGen_GRPO_7B")

headline = "AI writes entire novel; readers say it felt a little too human"
prompt = (
    "<|im_start|>system\n"
    "You are a comedy writer. Write one sharp, witty joke for the headline.\n<|im_end|>\n"
    f"<|im_start|>user\n{headline}<|im_end|>\n"
    "<|im_start|>assistant\n"
)
inputs  = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=120, temperature=0.9, top_p=0.95)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Citation

bibtex

@misc{ajayi2026humorgen,
  title         = {HumorGen: Cognitive Synergy for Humor Generation in Large Language
                   Models via Persona-Based Distillation},
  author        = {Ajayi, Edward and others},
  year          = {2026},
  eprint        = {2604.09629},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2604.09629}
}