PINO PIMT — Representation A/B (Morgan vs genuine OpenPOM vs OpenPOM+all-tags)
Representation ablation for the PINO Physics-Informed Mixture Transformer
(PIMT). Three arms trained on the identical corrected corpus
(empirical_dataset_v10_alltags.jsonl, molecule-disjoint split, seed-fixed)
differing only in the structural embedding block and (for the all-tags arm) the
readout width:
| Arm | Structural block | Input dim | Objective head | Checkpoint |
|---|
| Morgan fallback | 138-dim (128-bit Morgan + 10 physchem) | 151 | 138-dim | pimt_ab_morgan_v2.pt |
| Genuine OpenPOM | 256-dim OpenPOM v1.0.0 (ensemble mean) | 269 | 138-dim | pimt_ab_openpom_256_v2.pt |
| OpenPOM + all-tags | 256-dim OpenPOM | 269 | 575-dim tag | pimt_ab_pom_alltags_v2.pt |
Selected via --structural-source morgan|openpom_256|pom_alltags. CAS absent
from the genuine-POM asset receive a real Morgan block zero-padded to 256 so the
input dim is constant. The genuine-POM asset load fails closed (no silent
Morgan substitution).
v2 corrected-label readout (20 epochs, T4 GPU) — authoritative
Trained/evaluated on the v10 corpus with regenerated pyramid targets after a
label-corruption defect was found and fixed (see disclosure below). Evaluated
once per arm on the frozen benchmarks (labels evaluation_only):
| Metric | Morgan | Genuine OpenPOM | OpenPOM + all-tags |
|---|
| Final validation total loss (best epoch) | 0.4651 | 0.4357 | 0.3668 |
| Objective-head loss (best epoch) | 0.0402 | 0.0495 | 0.0830 |
| Substitution triplets (n=20, chance=0.5) | 0.35 | 0.45 | 0.35 |
| Prospective family-profile cosine (n=40) | 0.5167 | 0.4971 | n/a |
Full per-item detail: frozen_eval_morgan_v2.json,
frozen_eval_openpom_256_v2.json, frozen_eval_pom_alltags_v2.json
(sha256-pinned). Prospective family-profile agreement is only defined for the
138-dim readout, so it is n/a for the 575-dim all-tags arm.
Honest readout. The three arms are closely matched; no arm dominates.
Genuine OpenPOM edges substitution triplets (0.45 vs 0.35/0.35) and the all-tags
arm reaches the lowest validation loss, but all arms remain below chance (0.5)
on substitution triplets and prospective family agreement is moderate (~0.50).
These are genuine negative/mixed results, reported as such rather than spun as a
win. The corrected labels substantially improved objective-head convergence
(objective loss ~0.04–0.08 vs ~0.37 degenerate under the corrupt labels), which
was the primary purpose of the v2 retrain.
Superseded v1 readout (corrupt labels — lineage only, do not cite)
The first GPU run trained on the v9 corpus whose pyramid targets were silently
corrupted (30.2% of records collapsed to base-only; tier mass 4.05/2.53/16.67).
Its headline numbers (val 0.3959/0.3600, triplets 0.30/0.45, prospective
0.5378/0.6121) were computed against those defective targets and are superseded
by the v2 readout above. Retained here for provenance only.
Prior readouts (lineage, superseded)
- Model-free ablation (
representation_ablation_*): genuine OpenPOM beats
Morgan on every frozen predictive task (5-fold paired bootstrap).
- CPU run (6 epochs, n=8 triplets): OpenPOM ahead on all three — directional
only.
Label-corruption disclosure
A defect in the pyramid-target generator
(scripts/generate_pyramid_targets.py, now superseded) caused an OAV-decay
classifier to default 1,366 of 2,420 CAS to base-only, collapsing 30.2% of v9
pyramid_targets to the base tier. The defect was caught in pre-submission
review, the targets were regenerated with a corrected, width-agnostic fold
(scripts/regenerate_pyramid_targets_fold.py), producing the v10 corpus
(base-only 6.8%, tier mass 4.55/4.19/3.20), and all three arms were retrained
from scratch (this v2 readout). A float32 rounding artifact (1.00000012 > 1) in
49 all-tags records additionally trips the CUDA BCE kernel; BCE targets are
clamped to [0,1] in src/pino/train.py (621906d). The v9 corpus and its
derived results must not be used.
Lineage
- Corpus: 5,708 formulation/trajectory records; 604 pure single-molecule anchors;
molecule-disjoint train/validation (0 shared active CAS).
- Triplets: 20 discordant substitution triplets admitted (8 initial + 12 via
material_profiles trade-name mining), structural discordance + authoritative
preference provenance.
- Prospective: 40 preregistered in-silico formulas from a 51-material hobbyist
palette; intended family profiles sequestered (
labels.sequestered.json).
- Code:
mattbitzesty/pino-source-code. Demo Space: mattbitzesty/pino.
Known limitations
- Substitution-triplet accuracy is below chance (0.5) for all arms at n=20 —
a small-n, genuine negative.
- Prospective family agreement is in-silico only (no human sensory panel).
- The single-molecule objective head collapses out-of-domain to a "sweet"
attractor; it is intentionally not exposed in the demo and must not be used
for single-molecule descriptor lookup.
- The substantivity head is unsupervised (no ground-truth persistence labels).
License: MIT.
