You can deploy this model and start earning money today!

kabing/
gpt2-cbctg-prior

Name: gpt2-cbctg-prior
Availability: InStock

peft

controllable-text-generation

concept-bottleneck

compositional-generalization

lora

2608.22956

openai-community/gpt2

adapter

mit

Views

No views yet

gpt2-cbctg-prior

Concept-bottleneck controllable-generation modules for GPT-2 (124M), from the paper "The Illusion of Control: Why Bare Classifier Inversion Silently Fails in Concept-Bottleneck Text Generation" (EMNLP 2026).

Paper: arXiv:2608.22956

Primary backbone for Table 1 and for most appendix diagnostics (off-manifold distance, regulariser sweeps, the $k$-sample control, and the normalising-flow baseline).

This repository contains only our own trained parameters: the per-axis concept encoder MLPs, the per-axis classifier heads, the concept injector, the LoRA adapter, and the post-hoc label prior. No base-model weights are included; download gpt2 separately.

Each split directory holds one trained checkpoint.

Path	Description
`hold-out/`	Fyelp Hold-Out idx=-0 split (39 seen / 1 unseen configuration)
`acd/`	Fyelp ACD split (half of all configurations held out)

Within each split directory:

File	Description
`config.json`	Architecture and training configuration
`full_model_best.pt`	Concept encoder MLPs, classifier heads, injector, internal prior
`best_lora_adapter.pt`	LoRA adapter weights (rank 8, alpha 16); optimiser state stripped
`label_prior.pt`	Post-hoc label-conditioned prior $g_\gamma$ (the paper's recommended z-source)
`retrieval_prior.pt`	Nearest-seen-configuration retrieval baseline (Appendix L), where available
`label_prior_flow.pt`	Conditional normalising-flow baseline (Appendix K), where available
`label_prior_factorized.pt`	Factorised per-axis prior (Appendix N), where available
`clsinv_reg_*.pt`	Manifold-regulariser statistics for the inversion sweeps (Appendix I), where available

Architecture

Concept code: 4 axes (cuisine, gender, sentiment, tense), 32 dimensions per axis
Injector: AdaLN-zero, injected at every block
Generator: GPT-2 (124M) with LoRA (rank 8, alpha 16) on the attention projections
Prior $g_\gamma$: one hidden layer, 128 GELU units, fitted post-hoc in under 30 seconds

Usage

Clone the code repository and point the generation script at a downloaded checkpoint directory:

bash

git clone https://github.com/BiancaBing/cbctg-illusion-of-control
cd cbctg-illusion-of-control

huggingface-cli download kabing/gpt2-cbctg-prior --local-dir ckpt/gpt2-cbctg-prior

python evaluation/generate_v3.py \
    --checkpoint ckpt/gpt2-cbctg-prior/hold-out/full_model_best.pt \
    --data_dir   data/fyelp_hold-out_idx0_4ax \
    --output_dir out/ \
    --model_name gpt2-cbctg-prior_prior \
    --z_source prior \
    --prior_path ckpt/gpt2-cbctg-prior/hold-out/label_prior.pt

Swap --z_source prior for oracle (reference-text encoding) or mode_b (classifier inversion) to reproduce the other two protocols compared in the paper. Full reproduction instructions are in the code repository.

Licence

GPT-2 is released under the MIT License.

Our released parameters (encoder MLPs, classifier heads, injector, LoRA adapter, label prior) are made available for research use under the same terms as the corresponding base model.

Citation

bibtex

@inproceedings{bing2026illusion,
  title     = {The Illusion of Control: Why Bare Classifier Inversion
               Silently Fails in Concept-Bottleneck Text Generation},
  author    = {Bing, Qi and Shao, Xiaowei},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in
               Natural Language Processing (EMNLP)},
  year      = {2026},
  eprint    = {2608.22956},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url       = {https://arxiv.org/abs/2608.22956}
}

kabing/gpt2-cbctg-prior