Token-level social-bias detector built on gpt2 (causal decoder). Given a sentence, it tags each token with one of four bias categories following a 7-label BIO scheme, highlighting which words carry bias.

Part of the Attention Atlas project (a master's thesis on interpretable bias detection through transformer attention). It provides a causal-model counterpart to the BERT detectors, for studying how bias signals differ between bidirectional and autoregressive architectures.

Base model: gpt2
Task: multi-label token classification (per-token sigmoid, thresholded)
Attention: standard causal mask (kept at inference — the model was fine-tuned causally)
Language: English
Related models: pinthoz/gus-net-gpt2-medium, pinthoz/gus-net-bert, pinthoz/gus-net-bert-large

Label scheme

Index	Label	Category
0	O	none
1	B-STEREO	Stereotype (span start)
2	I-STEREO	Stereotype (span inside)
3	B-GEN	Generalisation (span start)
4	I-GEN	Generalisation (span inside)
5	B-UNFAIR	Unfair language (span start)
6	I-UNFAIR	Unfair language (span inside)

GEN — a blanket generalisation about a group.
UNFAIR — unfair / disparaging language toward a group.
STEREO — a stereotype attributed to a group.

Important: multi-label + per-label thresholds

Outputs are per-token sigmoid probabilities (multi-label), not a softmax. F1-optimised thresholds (order [O, B-STEREO, I-STEREO, B-GEN, I-GEN, B-UNFAIR, I-UNFAIR]):

[0.4250, 0.3977, 0.3500, 0.3617, 0.3913, 0.3278, 0.4174]

A flat 0.5 threshold will mis-detect bias — use the values above. They also ship with the model as optimized_thresholds.npy. This checkpoint's probabilities are scaled lower than a standard run (a side effect of the sparsity regulariser), so a 0.5 cut-off fires almost nothing.

Usage

GPT-2 has no [CLS]/[SEP]; the tokenizer needs add_prefix_space=True and a pad token. The first token is an attention-sink position — be cautious reading its scores.

python

import torch
from transformers import AutoTokenizer, GPT2ForTokenClassification

model_id = "pinthoz/gus-net-gpt2"
tok = AutoTokenizer.from_pretrained("gpt2", add_prefix_space=True)
tok.pad_token = tok.eos_token
model = GPT2ForTokenClassification.from_pretrained(model_id).eval()

CATEGORY_INDICES = {"STEREO": [1, 2], "GEN": [3, 4], "UNFAIR": [5, 6]}
THRESHOLDS = [0.4250, 0.3977, 0.3500, 0.3617, 0.3913, 0.3278, 0.4174]

text = "Women are naturally worse at driving."
enc = tok(text, return_tensors="pt")
with torch.no_grad():
    probs = torch.sigmoid(model(input_ids=enc["input_ids"],
                                attention_mask=enc["attention_mask"]).logits)[0]

tokens = tok.convert_ids_to_tokens(enc["input_ids"][0])
for i, tokn in enumerate(tokens):
    fired = {cat: float(probs[i, idxs].max())
             for cat, idxs in CATEGORY_INDICES.items()
             if any(probs[i, j] > THRESHOLDS[j] for j in idxs)}
    if fired:
        print(f"{tokn:15s} -> {fired}")

Training data

Fine-tuned on the GUS-Net dataset — a token-level social-bias corpus annotated for Generalisations, Unfairness and Stereotypes (ethical-spectacle/gus-dataset-v1).

This checkpoint is the sparsity-regularised training run (a penalty that concentrates attention mass on fewer tokens), trained on the cleaned corpus.

Difference from the original GUS-Net dataset and models: in the raw corpus punctuation is mostly fused to the preceding word rather than tokenised separately, so a comma or full stop falling inside a labelled span inherits that span's categories — the sentence-final mark carries a bias label in 1,942 of the 3,739 sentences. The corpus used here splits each mark into a token of its own and labels it non-bias O, repairing the BIO sequence where the split interrupts a span, since punctuation is not a social-bias carrier. On a held-out sample this checkpoint labels the sentence-final mark as bias in 0 of 741 sentences (0.0 %), matching the cleaned gold.

Evaluation

StereoSet (intersentence split, 2123 examples)

Metric	Score
LMS (language-modeling score, higher is better)	77.04
SS (stereotype score, 50 = ideal)	51.15
ICAT (bias-adjusted quality)	75.26

Per-category SS: gender 57.85 · race 48.26 · religion 55.13 · profession 52.24.

Token classification (GUS-Net held-out test set)

Held-out partition (747 sentences) of the stratified cross-validation fold this checkpoint was trained against — StratifiedKFold(n_splits=5, shuffle=True, random_state=42), fold 4 — scored with the per-label thresholds above. Each category aggregates its B-/I- labels; the micro average covers the three bias categories and excludes the majority O class. Label alignment mirrors training (continuation subtokens labelled, with B- demoted to I-).

Category	Precision	Recall	F1
O (non-bias)	0.870	0.930	0.899
GEN	0.797	0.625	0.701
UNFAIR	0.484	0.437	0.460
STEREO	0.734	0.652	0.690
Micro-avg	0.723	0.617	0.666

The sparsity penalty costs recall. Measured against an otherwise identical run without the penalty, on the same corpus and the same fold, this checkpoint holds the same precision (0.723) but recalls less (0.617 vs 0.771), for ~8 points of micro-F1. That is the intended trade: LAMBDA_SPARSE concentrates attention mass on fewer tokens, yielding tighter, more conservative spans. Prefer pinthoz/gus-net-gpt2-medium (0.876 micro) if you want maximum detection rather than concentrated attention.

Attention calibration (Attention Atlas)

Empirical noise floors for reading this checkpoint's attention, derived from a permutation null over 5,444 sentences. They answer "is this head's behaviour distinguishable from chance?", and are what the dashboard uses to avoid over-reading attention. Useful if you build your own attention analysis on these weights; not needed for plain token classification.

Quantity	α=0.05 (p95)	α=0.01 (p99)
BAR (Bias Attention Ratio)	2.56	5.71
BSR (Bias Self-Reinforcement)	4.34	10.12
Ablation representation-impact	0.0033	0.0165

Top-K head-ablation elbow: 4 heads. A head only counts as bias-specialised above the BAR floor, and its ablation only counts as causal above the impact floor. Note these floors are a property of the specific weights, not of the architecture, so they must be re-derived if the checkpoint is retrained: across three retrains of this same repo id the impact floor moved from 0.0194 to 0.0465 to 0.0033. The sparsity penalty also shows up here, with a higher BAR/BSR floor than the BERT sibling.

Limitations & intended use

Research / auditing tool, not a content-moderation oracle. Predictions reflect a specific operationalisation of bias; subtle or context-dependent bias may be missed.
Causal masking means each token only sees left context, so span boundaries can differ from the BERT models.
English only.
Do not use for automated decisions about individuals.

Citation

If you use these models, please cite the GUS-Net dataset and benchmark:

bibtex

@article{powers2024gusnet,
  title   = {GUS-Net: Social Bias Classification in Text with Generalizations, Unfairness, and Stereotypes},
  author  = {Powers, Maximus and Raza, Shaina and Chang, Alex and Riaz, Rehana and Mavani, Umang and Jonala, Harshitha Reddy and Tiwari, Ansh and Wei, Hua},
  journal = {arXiv preprint arXiv:2410.08388},
  year    = {2024}
}

License

Weights released under MIT (matching the gpt2 base model). The Attention Atlas code is MIT-licensed.