Unlearned Checkpoint
| Field | Value |
|---|
| Unlearning method | CRISP |
| Base model | google/gemma-2-2b-it |
| Target concept | Baseball |
| Checkpoint type | LoRA Adapter |
| Rank / seed | 100 / 42 |
| Train eval protocol | mc |
Unlearning Configuration
Selected hyperparameters (from unlearned_checkpoints.json):
| Parameter | Value |
|---|
alpha | 50 |
delta_embed | 0 |
k_features | 5 |
k_features_embed | 0 |
layer_hi | 14 |
layer_lo | 4 |
layer_step | 2 |
lora_rank | 4 |
lr | 0.0001 |
n_tokens_edited | 0 |
num_epochs | 2 |
Primary Unlearning Metrics (held-out test, MC protocol)
Headline scores used for checkpoint selection:
| Metric | Train (after unlearning) | Test (after unlearning) |
|---|
| Efficacy | 0.746 | 0.359 |
| Specificity | 0.9 | 0.778 |
| Harmonic mean | 0.816 | 0.491 |
| Relearning QA (MC) | — | 0.62 |
Full Evaluation (baseline → unlearned)
From evaluation/score_comparison.csv:
| Metric | Baseline (train) | After unlearn (train) | Baseline (test) | After unlearn (test) |
|---|
| QA accuracy | 0.84 | 0.4 | 0.64 | 0.5 |
| QA fraction | 1 | 0.254 | 1 | 0.641 |
| SimDom accuracy | 0.68 | 0.66 | 0.74 | 0.58 |
| SimDom fraction | 1 | 0.953 | 1 | 0.673 |
| MMLU accuracy | 0.52 | 0.48 | 0.551 | 0.527 |
| MMLU fraction | 1 | 0.852 | 1 | 0.92 |
Files in This Repository
| File | Description |
|---|
unlearned_checkpoints.json | Checkpoint metadata & hyperparameters |
evaluation/evaluation_summary.json | Full evaluation payload (train/test/relearning) |
evaluation/score_comparison.csv | Baseline vs. unlearned comparison table |
