The Deep-Fake-Detector-v2-Model is a state-of-the-art deep learning model designed to detect deepfake images. It leverages the Vision Transformer (ViT) architecture, specifically the google/vit-base-patch16-224-in21k model, fine-tuned on a dataset of real and deepfake images. The model is trained to classify images as either "Realism" or "Deepfake" with high accuracy, making it a powerful tool for detecting manipulated media.

Classification report:

              precision    recall  f1-score   support

     Realism     0.9683    0.8708    0.9170     28001
    Deepfake     0.8826    0.9715    0.9249     28000

    accuracy                         0.9212     56001
   macro avg     0.9255    0.9212    0.9210     56001
weighted avg     0.9255    0.9212    0.9210     56001

Confusion Matrix:

[[True Positives, False Negatives],
 [False Positives, True Negatives]]

Update : The previous model checkpoint was obtained using a smaller classification dataset. Although it performed well in evaluation scores, its real-time performance was average due to limited variations in the training set. The new update includes a larger dataset to improve the detection of fake images.

Repository	Link
Deep Fake Detector v2 Model	GitHub Repository

Key Features

Architecture: Vision Transformer (ViT) - google/vit-base-patch16-224-in21k.
Input: RGB images resized to 224x224 pixels.
Output: Binary classification ("Realism" or "Deepfake").
Training Dataset: A curated dataset of real and deepfake images.
Fine-Tuning: The model is fine-tuned using Hugging Face's Trainer API with advanced data augmentation techniques.
Performance: Achieves high accuracy and F1 score on validation and test datasets.

Model Architecture

The model is based on the Vision Transformer (ViT), which treats images as sequences of patches and applies a transformer encoder to learn spatial relationships. Key components include:

Patch Embedding: Divides the input image into fixed-size patches (16x16 pixels).
Transformer Encoder: Processes patch embeddings using multi-head self-attention mechanisms.
Classification Head: A fully connected layer for binary classification.

Training Details

Optimizer: AdamW with a learning rate of 1e-6.
Batch Size: 32 for training, 8 for evaluation.
Epochs: 2.
Data Augmentation:
- Random rotation (±90 degrees).
- Random sharpness adjustment.
- Random resizing and cropping.
Loss Function: Cross-Entropy Loss.
Evaluation Metrics: Accuracy, F1 Score, and Confusion Matrix.

Inference with Hugging Face Pipeline

python

from transformers import pipeline

# Load the model
pipe = pipeline('image-classification', model="prithivMLmods/Deep-Fake-Detector-v2-Model", device=0)

# Predict on an image
result = pipe("path_to_image.jpg")
print(result)

Inference with PyTorch

python

from transformers import ViTForImageClassification, ViTImageProcessor
from PIL import Image
import torch

# Load the model and processor
model = ViTForImageClassification.from_pretrained("prithivMLmods/Deep-Fake-Detector-v2-Model")
processor = ViTImageProcessor.from_pretrained("prithivMLmods/Deep-Fake-Detector-v2-Model")

# Load and preprocess the image
image = Image.open("path_to_image.jpg").convert("RGB")
inputs = processor(images=image, return_tensors="pt")

# Perform inference
with torch.no_grad():
    outputs = model(**inputs)
    logits = outputs.logits
    predicted_class = torch.argmax(logits, dim=1).item()

# Map class index to label
label = model.config.id2label[predicted_class]
print(f"Predicted Label: {label}")

Dataset

The model is fine-tuned on the dataset, which contains:

Real Images: Authentic images of human faces.
Fake Images: Deepfake images generated using advanced AI techniques.

Limitations

The model is trained on a specific dataset and may not generalize well to other deepfake datasets or domains.

Performance may degrade on low-resolution or heavily compressed images.
The model is designed for image classification and does not detect deepfake videos directly.

Ethical Considerations

Misuse: This model should not be used for malicious purposes, such as creating or spreading deepfakes. Bias: The model may inherit biases from the training dataset. Care should be taken to ensure fairness and inclusivity. Transparency: Users should be informed when deepfake detection tools are used to analyze their content.

Future Work

Extend the model to detect deepfake videos.
Improve generalization by training on larger and more diverse datasets.
Incorporate explainability techniques to provide insights into model predictions.

Citation

bibtex

@misc{Deep-Fake-Detector-v2-Model,
  author = {prithivMLmods},
  title = {Deep-Fake-Detector-v2-Model},
  initial = {21 Mar 2024},
  second_updated = {31 Jan 2025},
  latest_updated = {02 Feb 2025}
}