You can deploy this model and start earning money today!

arturoornelasb/
triadic-gpt2-medium

discrete-representations

Views

No views yet

GitHub: arturoornelasb/triadic-microgpt | Paper: Emergent Duality (2026) | PyPI: triadic-head

Triadic GPT-2 Medium (355M + 72-bit triadic head)

GPT-2 Medium (355M parameters) augmented with a triadic projection head that produces discrete prime-factor semantic signatures alongside standard next-token predictions.

The backbone is frozen during training -- only the 73,728-parameter triadic head is trained, preserving the base model's language ability (PPL identical to baseline).

Checkpoints

Two training variants are included:

Variant	Bit Accuracy	Dead Bits	Description
`ternary_v2/model_best.pt`	91.3%	26	Ternary activation (v2)
`sparsity_v2/model_best.pt`	91.2%	26	Sparsity-regularized (v2)

Architecture

Base model: GPT-2 Medium (355M params, frozen)
Triadic head: Linear projection 1024 -> 72 bits
Activation: iFSQ (2*sigmoid(1.6x) - 1)
Trainable params: 73,728 (0.02% of model)

Training

Data: WikiText-103 (~100M tokens)
Loss: Language + 0.05 * (triadic + 2supervised + 5subsumption)
Optimizer: AdamW (3e-4, beta 0.9/0.95)
Schedule: Cosine decay, 50% triadic warmup
Steps: 50,000
Hardware: NVIDIA RTX 4060 Ti 16GB

Key Results

Bit accuracy: ~91% (72-bit semantic signatures)
Perplexity: 31.95 (identical to GPT-2 Medium baseline)
Regla de Tres: 0.996 mean cosine (near-perfect relational structure)
Phase transition: Significant at step ~6000 (p=0.0004)

Intended Use

Research on neurosymbolic AI, discrete representation learning, and interpretable semantic structures. The triadic head maps continuous embeddings to algebraic prime-factor signatures enabling exact logical operations (subsumption, composition, gap analysis) impossible under cosine similarity.

Limitations

D-A-N classifications used for training were self-evaluated by the author; independent rater validation is pending.
Only 5 of 13 dual axes show significant anti-correlation after FDR correction.
Layer ordering is not recovered in neural representations (p=0.71).

Files

ternary_v2/
  model_best.pt          # Checkpoint (1.4GB)
  training_log.csv       # Per-step metrics
sparsity_v2/
  model_best.pt          # Checkpoint (1.4GB)
  training_log.csv       # Per-step metrics
data/
  primitivos.json        # 72 semantic primitives (DAG, primes, bits)
  anclas.json            # Training anchors
  anclas_v2.json         # Extended anchors
LICENSE                  # BUSL-1.1

Related Models

triadic-gpt-40m -- End-to-end 40M param model (unfrozen backbone)

Usage

python

import torch
checkpoint = torch.load("ternary_v2/model_best.pt", map_location="cpu")
# Contains model state dict + triadic head weights

For the full training/inference pipeline, install triadic-head:

pip install triadic-head

Citation

bibtex

@article{ornelas2026emergent,
  title={Emergent Duality: 14+ Fundamental Dualities Across 6 Algebraic Layers},
  author={Ornelas Brand, J. Arturo},
  year={2026}
}

License

BUSL-1.1 -- free for individuals, academic, and non-profit use. Commercial production use requires a participation agreement. See COMMERCIAL.md.

arturoornelasb/triadic-gpt2-medium