TwinCar Classifier
TwinCar is a vehicle make, model, and auxiliary year recognition project developed for the Brainster Data Science Academy Machine Learning Final Project.
The final deployed model is an EfficientNet-B3 classifier fine-tuned on Stanford Cars. It predicts one of 196 fine-grained Stanford Cars classes, then derives vehicle make, model, and year from the predicted class metadata.
Final deployed checkpoint
- Checkpoint file:
efficientnet_b3_stanford300_augv2_best.pt
- Architecture: EfficientNet-B3
- Input size: 300 px
- Classes: 196 Stanford Cars fine-grained classes
- Training data: Stanford Cars training split
- Training augmentation: augmentation v2
- Framework: PyTorch + timm
- Checkpoint manifest:
checkpoint_manifest.json
- Current status: final deployed candidate
The model repo also keeps older checkpoints for comparison and rollback:
efficientnet_b3_stanford300_best.pt — previous EfficientNet-B3 checkpointbest.pt — older ResNet18 baseline checkpoint
What the model predicts
The model directly predicts a fine-grained class, for example:
From that fine-grained prediction, the system derives:
- make — e.g.
Dodge
- model — e.g.
Charger SRT-8
- year — e.g.
2009
Year is included as an auxiliary output, but it is not predicted by a separate year-regression or year-classification head. It is derived from the predicted fine-grained class metadata.
Validation results
Final quantitative comparison is reported on the locked Stanford validation split using the same evaluation protocol for all compared models.
| Transform | Fine acc | Make acc | Model acc | Year acc | Top-3 acc | Top-5 acc |
|---|
| clean | 0.7864 | 0.8692 | 0.7925 | 0.8913 | 0.9196 | 0.9521 |
| robust_light | 0.7882 | 0.8680 | 0.7944 | 0.8956 | 0.9159 | 0.9490 |
| robust_hard | 0.6839 | 0.7778 | 0.6900 | 0.8355 | 0.8600 | 0.9055 |
| robust_occlusion | 0.6317 | 0.7317 | 0.6366 | 0.8048 | 0.8060 | 0.8600 |
Compared with the earlier EfficientNet-B3 candidate (no augmentation v2), this model improves clean fine accuracy by +1.6 pts (0.770 → 0.786) and
robustness substantially — robust_hard +14 pts (0.543 → 0.684) and robust_occlusion +13 pts (0.502 → 0.632). Full comparison: the GitHub experiment report.
Robustness evaluation
The final model was evaluated under multiple image transforms:
- clean — standard validation preprocessing
- robust_light — mild production-like perturbations
- robust_hard — stronger blur, lighting, color, and geometric perturbations
- robust_occlusion — synthetic occlusion/erasing stress test
These tests are not a replacement for real-world field validation, but they quantify how the model behaves under controlled distribution shifts.
CompCars status
CompCars was inspected and used for external validation and reconnaissance, but it was not blindly merged into final training.
The main reason is that Stanford Cars and CompCars have a significant domain and label-distribution gap:
- Stanford Cars is a clean fine-grained benchmark with 196 make/model/year classes.
- CompCars contains different image domains and different make/model taxonomies.
- Exact Stanford Cars ↔ CompCars make/model/year overlap was too small and biased for safe blind merging.
- Make-level external validation on CompCars dropped to ~28% (vs ~0.87 make accuracy in-domain), confirming a large cross-domain shift.
This confirmed that CompCars integration is a domain adaptation problem, not a simple data-merge task.
Future work should build a verified Stanford Cars ↔ CompCars alias map, train on a controlled filtered subset, and validate on a true cross-domain holdout.
Held-out Stanford test status
The local Stanford Cars test images were available and were used for qualitative API/demo smoke testing.
However, the available cars_test_annos.mat file contained only bounding boxes and filenames:
1bbox_x1
2bbox_y1
3bbox_x2
4bbox_y2
5fname
It did not include class labels.
The provided Kaggle mirror, eduardo4jesus/stanford-cars-dataset, was also checked. It included:
1cars_meta.mat
2cars_train_annos.mat
3cars_test_annos.mat
but did not provide:
1cars_test_annos_withlabels.mat
2cars_annos.mat
Final quantitative model comparison is therefore reported on the locked Stanford validation split using an identical protocol and seed for every compared model. This keeps model-to-model deltas valid. A labeled held-out Stanford test evaluation would be a straightforward extension if cars_test_annos_withlabels.mat is obtained.
Demo
The model is deployed in a Hugging Face Space:
The Space supports:
- image upload
- webcam/clipboard input
- full-image prediction
- make/model/year display
- top-k predictions
- optional YOLO crop comparison mode
The YOLO cropper is experimental and default-off. The official prediction remains the full-image EfficientNet-B3 prediction.
API and code
Project repository:
The GitHub repo includes:
- reusable Python package
- FastAPI inference endpoint
- Gradio demo app
- batch prediction script
- training and evaluation scripts
- robust evaluation reports
- CI with linting and tests
W&B
Training and experiment tracking:
Intended use
This model is intended for educational and prototype-level vehicle recognition experiments, especially make/model classification from car images similar to Stanford Cars.
Appropriate uses:
- fine-grained vehicle make/model recognition demo
- model comparison and robustness analysis
- prototype vehicle inspection workflow
- academic/academy project demonstration
Data note: weights are trained on the Stanford Cars dataset (research/educational use); the MIT license covers the project code.
Use of the weights should respect the Stanford Cars dataset terms.
Limitations
- The final model is trained on Stanford Cars, not on real drone/robot production footage.
- CompCars showed a strong domain gap and was not blindly merged into final training.
- True top-down drone views remain out-of-distribution.
- Robustness tests use controlled synthetic perturbations, not full real-world field validation.
- Year is derived from the fine-grained class label metadata, not learned as an independent year model.
- The system assumes the uploaded image contains a vehicle.
- Strong non-car/out-of-distribution rejection is not implemented yet.
- Similar models and years can be confused because fine-grained vehicle classification often depends on subtle visual details.
Future work
- Build a verified Stanford Cars ↔ CompCars alias map.
- Fine-tune on a controlled CompCars surveillance subset.
- Add real production/drone/robot images for target-domain validation.
- Add non-car/out-of-distribution rejection.
- Add detector-assisted preprocessing as a validated default only if it improves real metrics.
- Explore multi-head prediction for make/model/year if independent outputs become necessary.
