This project implements a normal-only image anomaly detection pipeline using three complementary feature-extraction branches:
DINOv2 — captures global structure and semantic representation.
ConvNeXt — captures hierarchical and multi-scale visual patterns.
WideResNet50 + PatchCore-style scoring — focuses on local patch-level defects.
The system is designed for a one-class / unsupervised anomaly detection setting, where the training data contains only normal images.
Instead of training a conventional binary classifier, the pipeline learns the feature distribution of normal samples and assigns an anomaly score to unseen images based on how different they are from the learned normal distribution.
Pipeline
text
1 Input Image
2 |
3 +-----------------+------------------+
4 | | |
5 v v v
6 DINOv2 ConvNeXt WideResNet50
7 | | |
8 Global / semantic Multi-scale Local patch
9 features features features
10 | | |
11 PCA PCA PatchCore memory
12 | | |
13 kNN kNN Nearest normal patch
14 | | |
15 DINO score ConvNeXt score PatchCore score
16 | | |
17 +-----------------+------------------+
18 |
19 Robust score normalization
20 |
21 v
22 Equal-score fusion
23 |
24 v
25 Category-wise threshold
26 |
27 Normal / Anomaly
where each branch score is robustly normalized before fusion.
Why Three Branches?
DINOv2
DINOv2 is used for high-level and structural representation. It is useful for detecting anomalies such as:
abnormal object shape,
missing or extra components,
structural changes,
unusual object arrangement,
global appearance deviations.
The implementation uses intermediate transformer representations instead of relying only on the final embedding.
ConvNeXt
ConvNeXt provides hierarchical CNN features at multiple stages. It is useful for:
texture anomalies,
surface changes,
local appearance variations,
medium-scale structural differences,
abnormal visual patterns.
Features from several ConvNeXt stages are pooled and concatenated before dimensionality reduction.
WideResNet50 + PatchCore-style Branch
The third branch focuses on local defects. Intermediate WideResNet50 feature maps are converted into patch embeddings and stored in a normal patch memory bank.
For each test patch, the model searches for the nearest normal patch. This branch is especially useful for detecting:
scratches,
small holes,
local deformations,
spots,
tiny missing parts,
localized texture defects.
The image-level PatchCore score is calculated from the most anomalous local patches rather than a single maximum value.
Normal-Only Training Strategy
The pretrained backbones are frozen in the baseline implementation. There is no conventional neural-network training stage.
text
1Normal training images
2 |
3 v
4Feature extraction
5 |
6 v
7Normal feature memory / distribution
8 |
9 v
10Distance-based anomaly scoring
This keeps the baseline focused on learning the distribution of normal data and reduces the risk of overfitting artificial anomalies.
5-Fold Out-of-Fold Validation
Because the training set contains only normal images, the project uses 5-fold out-of-fold (OOF) validation for each category.
text
1Normal images
2 |
3 v
45 folds
5 |
6 +--> 80% normal images -> build normal memory
7 |
8 +--> 20% unseen normal images -> OOF normal validation
The held-out normal images are never included in the memory used to score them. This provides a more realistic estimate of anomaly scores for unseen normal samples and avoids self-neighbor leakage in kNN-based scoring.
Synthetic Anomaly Validation
Synthetic anomalies are generated only from held-out normal images. They are used as a validation and stress-testing tool, not as real anomaly ground truth.
Implemented corruption types include:
CutPaste,
local masking,
patch duplication,
local blur,
synthetic scratches.
For each fold:
text
180% normal
2 |
3 +--> build normal model
4520% held-out normal
6 |
7 +--> original image --------> normal validation
8 |
9 +--> corrupted copy --------> synthetic anomaly validation
Synthetic validation is useful for comparing feature extractors, fusion strategies, and threshold robustness, but it is not assumed to perfectly represent hidden real anomalies.
Feature Processing
DINOv2 and ConvNeXt
For image-level features:
Extract pretrained features.
Fit PCA using normal training features only.
L2-normalize the reduced embeddings.
Build a k-nearest-neighbor normal memory.
Compute anomaly score from the mean distance to the nearest normal neighbors.
Equal weighting is intentionally used as the default because learned fusion weights can easily overfit synthetic validation data or a public leaderboard.
Future fusion experiments may include:
weighted averaging,
logistic-regression stacking,
leave-one-branch-out ablation,
category-specific fusion.
Category-Wise Thresholding
Each category is treated independently. A separate threshold is estimated from the normal anomaly-score distribution:
[
\tau_c = Q_p(S^{normal}_c)
]
where Q_p is a selected percentile.
The default configuration uses the 97.5th percentile.
The loader recursively scans the images/ directory and matches each image filename stem to its sample_id. The same mapping strategy is used for public and private test sets.
A branch should not be selected only because it performs well on one synthetic anomaly type or one public submission.
Recommended Experiment Order
Experiment
DINO
ConvNeXt
PatchCore
Fusion
E1
✓
Single
E2
✓
Single
E3
✓
Single
E4
✓
✓
Mean
E5
✓
✓
Mean
E6
✓
✓
Mean
E7
✓
✓
✓
Equal Mean
E8
✓
✓
✓
Learned / Weighted
The equal-weight fusion should be established as a stable baseline before introducing learned stacking weights.
Avoiding Public-Leaderboard Overfitting
Repeatedly changing thresholds, fusion weights, feature dimensions, kNN parameters, or synthetic corruption settings based only on public leaderboard feedback can lead to overfitting.
The recommended workflow is:
text
1Train-normal data
2 |
3 v
4OOF normal validation
5 +
6Synthetic stress testing
7 |
8 v
9Select stable configuration
10 |
11 v
12Freeze model and thresholds
13 |
14 v
15Public / Private inference
The public leaderboard should be treated as an external check rather than the primary source of hyperparameter optimization.
Possible Future Improvements
Potential extensions include:
leave-one-corruption-out validation,
multi-scale DINO patch features,
larger DINOv2 backbones,
larger ConvNeXt variants,
improved PatchCore coreset sampling,
Mahalanobis anomaly scoring,
score-level stacking,
category-specific branch weighting,
self-supervised fine-tuning on normal images,
contrastive learning with normal augmentations and synthetic anomalies.
Any learned fusion or fine-tuning strategy should be validated carefully to avoid overfitting synthetic anomalies.
This helps make feature extraction, OOF validation, synthetic corruption generation, and final predictions reproducible.
Summary
This project follows a simple principle:
Learn what normal looks like, then detect samples that deviate from the learned normal representation.
DINOv2, ConvNeXt, and PatchCore provide complementary views of image normality:
DINOv2 captures global structure,
ConvNeXt captures hierarchical texture and appearance,
PatchCore captures localized defects.
Their anomaly scores are normalized and fused to form a final category-specific anomaly decision without requiring labeled anomaly samples during training.
