PointCal-C — selective zero-shot 3D recognition under corruption
A post-hoc calibration and abstention head on top of a frozen OpenCLIP
ViT-B/32, for zero-shot 3D point-cloud recognition under corruption, plus the
complete evaluation artifacts backing it.
The backbone is never trained. laion/CLIP-ViT-B-32-laion2B-s34B-b79K is
loaded frozen, in eval mode, with requires_grad_(False).
The fitted model is four scalars, in runs/full/results/calibration.json,
fit on clean calibration objects only — corrupted labels, corruption identity
and severity never touch a fitted parameter:
parameter
value
role
temperature
1.2309
divides logits before softmax
bias
-1.5723
combined-score intercept
weight_confidence
+1.7071
on logit(p_calibrated)
weight_disagreement
+4.9158
on cross-view disagreement d
Combined score: sigmoid(bias + w_conf * logit(p_cal) + w_dis * d), where d
is 1 - mean pairwise Jensen-Shannon divergence across the six views.
Converged, unclamped, not degenerate. Fit in 0.94 s on CPU.
How inference works
point cloud (N,1024,3)
-> centroid-center, unit-radius normalize
-> 6 orthographic cameras [front right back left top bottom]
-> 64x64 splat raster + nearest-depth z-buffer -> upsample to 224x224
-> frozen OpenCLIP ViT-B/32 image encoder
-> per-view logits vs a fixed 8-template prompt ensemble <-- cached here
-> mean over views -> the prediction
-> four confidence scores over that one prediction
All four confidence methods (msp, temperature, disagreement, combined)
rank the same predictions. A confidence method that changed a prediction
would be a different classifier, and assert_predictions_unchanged raises if
one ever does.
Results
All 75 corrupted conditions pooled against clean, 1975 evaluation objects,
95% bootstrap intervals resampled over base object IDs:
accuracy
ECE (MSP)
ECE (combined)
AURC (MSP)
AURC (combined)
clean
0.2896
0.1160
0.0245
0.4339
0.4147
corrupted
0.2354
0.1416
0.0229
0.5217
0.5008
Accuracy degrades under corruption and MSP calibration degrades with it. The
clean-fit combined score holds ECE near 0.023 under shift and lowers AURC
by 0.021.
The ECE improvement is large and consistent. The AURC gain is small and
should be read against the intervals in results.csv, not as a headline. This
is a reliability audit and a bounded abstention baseline, not a
state-of-the-art claim.
Of the four pre-registered hypotheses, three are mechanically checked. H2 and H3
resolve True. H1 does not hold strictly: accuracy, NLL, Brier and AURC all
worsen monotonically with severity, but ECE dips at severities 3→4 and again
from clean→1. A paired object-grouped bootstrap places both reversals within
noise of zero, so H1 is supported in direction but not as strict monotonicity.
H4 is read off the family-scope rows rather than reduced to a verdict.
Repository contents
The expensive artifact is runs/*/logits/ — per-view logits for every
condition under both prompt modes. Every metric, ablation, bootstrap interval
and figure here was computed on CPU from those caches, so any re-analysis is
free: a different aggregation, disagreement statistic, coverage level or
bootstrap depth costs no GPU time, because CLIP never runs again.
runs/<tier>/
logits/<condition>.npz per-view logits, both prompt modes (~1.4 MB each)
results/calibration.json the fitted scalars above
results/results.csv|.json metrics table with intervals
results/ablations.csv view-count, prompt-mode, disagreement ablations
results/predictions.npz per-sample confidences and correctness
results/results_summary.md auto-generated tables + hypothesis checks
figures/fig1..fig4 accuracy, ECE, risk-coverage, cost
provenance/ data manifest, run manifest, split audit
ledger_inference.json measured GPU-hours, dollars, throughput, memory
artifacts/split.json the frozen calibration/evaluation object split
env/requirements.lock.txt exact resolved environment (171 packages)
Tiers: xs (5 conditions, 100 objects), s (13 conditions), full (all 76 =
clean + 15 corruptions x 5 severities).
Training / evaluation data
Evaluated on ModelNet40-C (2468 objects, 40 classes, 15 corruption types,
5 severities). The calibration scalars are fit on the clean split of 493
calibration objects; 1975 evaluation objects are held out.
The split is over base object IDs, not samples, and every corruption array
is verified row-aligned with the clean array, so holding an object out of
calibration holds it out under all 76 conditions. Audited: overlap: 0,
0 row-misaligned conditions, all 40 classes present on both sides.
Compute
tier
conditions
GPU-hours
USD
views/s
peak VRAM
xs
5
0.001
0.0003
1091
2.5 GB
s
13
0.015
0.004
2816
12.1 GB
full
76
0.089
0.025
2808
12.1 GB
0.105 GPU-hours and about $0.03 of inference in total, on a single Runpod
RTX 4000 Ada at $0.28/hr (2026-08-16).
Known defects
See ERRATUM.md for the full list. In short: the ledger block
embedded in runs/full/provenance/run_manifest.json is wrong — it records a
cache-hit re-invocation, not the real run. Trust ledger_inference.json,
which every figure and the compute table above are derived from. git_dirty in
that manifest is not meaningful either, because the dirtiness check counts
expected untracked artifacts.
Limitations
full uses 200 bootstrap replicates, not the preregistered 1000. The CPU
bootstrap dominated runtime (~2 h at 1000 vs ~5 min of GPU work). Deeper
intervals recompute from the cached logits at no GPU cost.
NLL and Brier require a distribution over all 40 classes, so they are reported
for msp and temperature only. For the two ranking scores they are null,
not a lookalike substitute.
delta_*_vs_clean rows in results.csv carry NaN intervals: degradation is
reported as point estimates without paired bootstrap intervals. Method
differences should likewise not be read from the marginal intervals alone.
Cross-view disagreement is prior art, and calibration degrading under 3D
corruption is a confirmation rather than a discovery.
Absolute accuracy is low (~0.29 clean). This is zero-shot transfer of a 2D
model to depth renders, with no 3D training whatsoever.
Severity is reported 1-indexed, matching the on-disk release.
Underlying ModelNet40 carries its own original terms
OpenCLIP — MIT · PointCal-C code — MIT
The multi-view depth-projection approach follows the PointCLIP paper (CVPR
2022); the projector is an independent reimplementation from the paper text,
with no PointCLIP repository code used.
