Proposes a self-supervised pre-train framework for vehicle-centric visual tasks.
Extends CGD-MAE with richer data analysis and an enhanced pre-training design.
Unifies masked-contrastive and CLIP-guided semantic objectives via feature fusion.
Ablation and qualitative results validate the proposed design.
Improves state-of-the-art vehicle-centric benchmarks in fine-tuning and linear-probe.

Abstract

In this work, we present VC-SCMAE, a Vehicle-Centric Semantic Contrastive-Guided Masked Autoencoder framework that distills knowledge from multimodal foundational models. Our approach extends MAE pre-training with contrastive guidance, combining masked image modeling with instance-level discrimination to produce more robust and transferable representations. On top of this discriminative backbone, we apply CLIP-style semantic distillation, leveraging a large-scale vehicle dataset (Automobile1M) and a visually grounded unpaired text corpus. Unlike conventional vision–language models that rely on aligned image–text pairs, our method transfers semantic knowledge from a pre-trained CLIP model without requiring explicit alignment. We further introduce specialized distillation losses that enhance open-vocabulary logits during vision-language distillation, thereby strengthening semantic alignment across modalities. Experiments demonstrate that VC-SCMAE effectively transfers to vehicle-specific downstream tasks via both linear probing and fine-tuning, unifying structural, discriminative, and semantic understanding within a single pre-training framework.

Citation

bibtex

@article{MARQUES2026131646,
    title = {VC-SCMAE: Vehicle-centric semantic contrastive-guided masked autoencoder},
    journal = {Expert Systems with Applications},
    volume = {315},
    pages = {131646},
    year = {2026},
    issn = {0957-4174},
    doi = {https://doi.org/10.1016/j.eswa.2026.131646},
    url = {https://www.sciencedirect.com/science/article/pii/S0957417426005592},
    author = {Alexandre Marques and Pedro Ferreira and Bruno Silva and Jorge Batista},
    keywords = {Masked autoencoders, Knowledge distillation, Contrastive learning, Self-supervised learning, Vehicle-centric pre-training, CLIP},
    abstract = {In this work, we present VC-SCMAE, a Vehicle-Centric Semantic Contrastive-Guided Masked Autoencoder framework that distills knowledge from multimodal foundational models. Our approach extends MAE pre-training with contrastive guidance, combining masked image modeling with instance-level discrimination to produce more robust and transferable representations. On top of this discriminative backbone, we apply CLIP-style semantic distillation, leveraging a large-scale vehicle dataset (Automobile1M) and a visually grounded unpaired text corpus. Unlike conventional vision–language models that rely on aligned image–text pairs, our method transfers semantic knowledge from a pre-trained CLIP model without requiring explicit alignment. We further introduce specialized distillation losses that enhance open-vocabulary logits during vision-language distillation, thereby strengthening semantic alignment across modalities. Experiments demonstrate that VC-SCMAE effectively transfers to vehicle-specific downstream tasks via both linear probing and fine-tuning, unifying structural, discriminative, and semantic understanding within a single pre-training framework.}
}

alexmaks02/VC-SCMAE

VC-SCMAE

Pipeline

Highlights

Abstract

Citation

alexmaks02/
VC-SCMAE