A 40M-parameter GPT language model trained end-to-end with a triadic projection head that produces discrete prime-factor signatures alongside standard next-token predictions.

Unlike the GPT-2 Medium variant, this model learns triadic representations end-to-end -- both the language model and the triadic head are trained jointly.

Checkpoint

Variant	Bit Accuracy	Dead Bits	Description
`model_best.pt`	93.0%	26	Best checkpoint (63-bit XL v2)

Architecture

Model: Custom GPT (12 layers, d_model=512, ~40M params)
Triadic head: Linear projection 512 -> 72 bits
Activation: iFSQ (2*sigmoid(1.6x) - 1)

Training

Data: WikiText-103 (~100M tokens)
Loss: Language + triadic (supervised + subsumption + diversity + InfoNCE)
Steps: 50,000
Hardware: NVIDIA RTX 4060 Ti 16GB

Key Results

Bit accuracy: 93.0%
Analogy verification: 98%
Signature uniqueness: 100%
Subsumption: 98.3% (higher than 355M model at 76.9%)

Intended Use

Research on neurosymbolic AI, discrete representation learning, and interpretable semantic structures. Demonstrates that a small language model can learn algebraic prime-factor signatures end-to-end without catastrophic forgetting.

Limitations

Trained on WikiText-103 only; generalization to other corpora not yet evaluated.
Higher subsumption rate (98.3%) compared to the larger 355M model (76.9%) suggests a scale-algebra tradeoff that is not yet understood.
D-A-N classifications are self-evaluated; independent rater validation pending.

Files

model_best.pt            # Checkpoint (153MB)
training_log.csv         # Per-step metrics
results.json             # Evaluation results
tokenizer.json           # Tokenizer
data/
  primitivos.json        # 72 semantic primitives (DAG, primes, bits)
  anclas.json            # Training anchors
  anclas_v2.json         # Extended anchors
LICENSE                  # BUSL-1.1

Related Models

triadic-gpt2-medium -- Frozen GPT-2 Medium 355M + triadic head

Usage

python

import torch
checkpoint = torch.load("model_best.pt", map_location="cpu")
# Contains full model state dict (language model + triadic head)

For the full training/inference pipeline and desktop UI, see triadic-microgpt.

Citation

bibtex

@article{ornelas2026triadicgpt,
  title={TriadicGPT: End-to-End Prime Factorization in a Generative Language Model},
  author={Ornelas Brand, J. Arturo},
  year={2026}
}

License

BUSL-1.1 -- free for individuals, academic, and non-profit use. Commercial production use requires a participation agreement. See COMMERCIAL.md.

arturoornelasb/triadic-gpt-40m

TriadicGPT (40M parameters + 72-bit triadic head)