skin_cancer_vit_classifier
Overview
This model is a Vision Transformer (ViT) fine-tuned on high-resolution dermatoscopic imagery. It is designed to assist in the preliminary classification of skin lesions into seven distinct diagnostic categories. It leverages self-attention mechanisms to identify subtle morphological patterns in skin tissue that are characteristic of specific malignancies.
Model Architecture
The model is based on the ViT-Base architecture (vit-base-patch16-224).
- Patches: The input image (224x224) is divided into 16x16 patches.
- Encoder: 12 layers of Transformer blocks with multi-head self-attention.
- Classification Head: A linear layer on the [CLS] token used to predict one of seven lesion types.
- Training: Fine-tuned using the HAM10000 dataset with heavy augmentation (rotation, flipping, and color jittering) to account for varying skin tones and lighting conditions.
Intended Use
- Clinical Decision Support: Providing a secondary "opinion" for dermatologists during screening.
- Medical Research: Analyzing feature importance maps to understand which visual cues correlate with specific lesion types.
- Educational Tools: Assisting medical students in identifying visual markers of BCC or Melanoma.
Limitations
- Not for Primary Diagnosis: This model is an experimental tool and should not replace professional medical diagnosis.
- Demographic Bias: Performance may vary significantly on skin types underrepresented in the training data (Fitzpatrick scales V and VI).
- Image Quality: Requires clear, focused dermatoscopic images. Standard smartphone photos without specialized lenses may lead to inaccurate results.
