You can deploy this model and start earning money today!

ARotting/
uncertainty-lens

Name: uncertainty-lens
Availability: InStock

uncertainty-quantification

deep-ensemble

calibration

out-of-distribution-detection

pytorch

apache-2.0

Views

No views yet

Uncertainty Lens

Uncertainty Lens trains five compact CNNs from independent initializations and turns their disagreement into a probabilistic diagnostic.

The project measures:

single-model versus ensemble accuracy;
negative log-likelihood, Brier score, and expected calibration error;
validation-only temperature scaling;
predictive entropy and mutual information on held-out digits versus uniform-noise and pixel-scrambled OOD inputs.

The OOD sets are synthetic stress tests, not evidence of real-world deployment safety.

Reproduce

powershell

uv run python projects/tiny-vision-foundry/prepare_data.py
uv run python projects/uncertainty-lens/train.py

Verified results

Metric	Single member	Five-member ensemble
Test accuracy	98.15%	99.26%
Negative log-likelihood	0.0551	0.0570
Brier score	0.02784	0.02498
Expected calibration error	0.02185	0.02547

Validation selected temperature 1.0, meaning temperature scaling provided no improvement and left ensemble metrics unchanged.

For 270 uniform-noise and 270 pixel-scrambled inputs, predictive entropy achieved 0.9771 ROC-AUC against clean held-out digits. Mean entropy rose from 0.105 on clean data to 1.113 on OOD data; mean mutual information rose from 0.018 to 0.462.

ARotting/uncertainty-lens

Uncertainty Lens

Reproduce

Verified results

ARotting/
uncertainty-lens