A 72-bit triadic projection head trained on top of frozen GPT-2 Medium (345M params). This is the production milestone that solved catastrophic forgetting (PPL = 31.95, identical to baseline) at the cost of only -1.7% bit accuracy vs. v4.

Paper DOI

Repo DOI

What is this?

This model adds a triadic head to GPT-2 Medium that projects language model embeddings into 72-bit binary codes representing ontological primitives from the Emergent Duality framework.

Each bit corresponds to one of 72 semantic primitives organized in 6 algebraic layers (Boolean, Fuzzy, Ordinal, Modal, Trivalent, Probabilistic). The head learns to map continuous embeddings to discrete binary codes that preserve ontological structure.

Key change vs. earlier versions: the GPT-2 backbone is frozen; only the 73,728-parameter triadic head trains (0.02% of model). This eliminates catastrophic forgetting that affected v1-v4 (PPL > 2,700) while preserving relational structure.

Files

File	Description
`best.pt`	Final trained weights (best checkpoint at 50K steps)
`gold_primitivos_72.json`	Gold standard: 72 core primitives, one bit each
`run_config.json`	Full training configuration
`results.json`	Final metrics
`training_log.csv`	Training curve (loss, accuracy per step)

Key Metrics

Metric	Value
Bit accuracy (test)	90.8%
Unique signatures	100% (72/72)
Coherence	0.818
PPL (general text)	31.95 (identical to GPT-2 baseline)
Active bits	4/72
Dead bits	68/72
Regla de tres (cosine)	0.997 (8 quadruples)
Trainable params	73,728 (0.02% of total)
Training time	4.3h on RTX 4060 Ti 16GB

Architecture

Base model: GPT-2 Medium (gpt2-medium, 355M params, frozen)
Triadic head: Linear(1024 → 72) with iFSQ activation ($2\sigma(1.6x) - 1$)
Loss: $\mathcal{L}\text{lang} + 0.05 \cdot (\mathcal{L}\text{tri} + 2\mathcal{L}\text{sup} + 5\mathcal{L}\text{sub})$

Usage

python

from transformers import GPT2Model, GPT2Tokenizer
import torch

# Load base GPT-2 Medium
tokenizer = GPT2Tokenizer.from_pretrained("gpt2-medium")
gpt2 = GPT2Model.from_pretrained("gpt2-medium")

# Load triadic head from this repo
head_state = torch.load("best.pt", map_location="cpu")
# See repo code for the TriadicHead module definition.

Full inference pipeline: see dualidademergente+reptimeline/model/triadic.py in the GitHub repo.

Training

bash

python train.py --bits 72 --gold-file gold_primitivos_72.json \
  --run-name gpt2_triadic_72_v5_frozen --freeze-base

Companion models

triadic-gpt2-medium-v6: Same architecture, scaled to 2,166 concepts (extended corpus).
triadic-gpt2-medium-v8: Deep head, 250K steps, base for V8 telescopic X-ray.
triadic-gptneo-125m-v9: GPT-Neo 125M / OpenWebText (cross-architecture replication).

Citation

bibtex

@software{ornelasbrand2026triadic,
  author = {Ornelas Brand, J. Arturo},
  title = {Triadic Emergent Duality: 14+ Candidate Dualities Across 6 Algebraic Layers},
  year = {2026},
  doi = {10.5281/zenodo.19374914},
  url = {https://github.com/arturoornelasb/triadic-emergent-duality}
}

License

Apache 2.0 for these weights. The companion code repository is BUSL-1.1.

arturoornelasb/triadic-gpt2-medium-v5

Triadic GPT-2 Medium (V5_frozen)