LUME (Language Model Unlearning Made Easy) provides OLMo models fine-tuned on 250 synthetic biographies containing PII (DOB, SSN, phone, email, address).

Evaluation metrics:

QA Accuracy: Fraction of PII recoverable via QA prompts (lower is better)
ROUGE-L: Overlap with memorized biographies
PPL: WikiText-2 perplexity

Results

Method	QA ↓	R-L ↓	PPL ↓	ΔPPL
Infected	100%	1.0	varies	---
PSTU	0%	~0.1	~clean	<2%

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Hodfa71/olmo-7b-lume-pstu-trim")
tokenizer = AutoTokenizer.from_pretrained("Hodfa71/olmo-7b-lume-pstu-trim")

Related Models

Citation

If you use this model, please cite our work on Per-Secret-Type Unlearning (PSTU).