You can deploy this model and start earning money today!

Hamzenium/
ViT-Deepfake-Classifier

Name: ViT-Deepfake-Classifier
Rating: 2.5 (6 reviews)

safetensors

vit

Views

No views yet

ViT Deepfake Detection Model

Model Description

This is a fine-tuned Vision Transformer (ViT) model for binary image classification to detect deepfake images. The model is based on google/vit-base-patch16-224-in21k and has been fine-tuned on the OpenForensics dataset to distinguish between real and fake (AI-generated/manipulated) images.

Model Details

Model Type: Vision Transformer (ViT) for Image Classification
Base Model: google/vit-base-patch16-224-in21k
Task: Binary Image Classification (Real vs Fake Detection)
Language: N/A (Computer Vision)
License: Apache 2.0

Intended Use

Primary Use Cases

Detecting AI-generated or manipulated images
Content moderation and verification
Research in deepfake detection
Media authenticity verification

Out-of-Scope Use

This model should not be used as the sole method for making critical decisions about content authenticity
Not intended for surveillance or privacy-invasive applications
May not generalize well to deepfake techniques not present in the training data

Training Data

The model was trained on the OpenForensics dataset with the following distribution:

Training Set: 16,000 images
Validation Set: 2000 images
Test Set: 2000 images

Images were preprocessed and transformed using ViTImageProcessor with standard normalization.

Training Procedure

Hyperparameters

python

Training Arguments:
- Batch Size: 24 per device
- Gradient Accumulation Steps: 1
- Mixed Precision: FP16
- Number of Epochs: 10
- Learning Rate: 3e-5
- Weight Decay: 0.02
- Warmup Ratio: 0.08
- LR Scheduler: Cosine
- Label Smoothing: 0.05
- Optimizer: AdamW (default)

Training Hardware

GPU: Tesla T4
Training Time: ~14 minutes for 10 epochs

Data Augmentation

Standard ViT preprocessing with normalization applied via ViTImageProcessor.

Performance

Validation Set Results (Best Epoch - Epoch 5)

Metric	Score
Accuracy	96.22%
F1 Score	96.22%
Precision	96.30%
Recall	96.22%

Test Set Results

Metric	Score
Accuracy	96.56%

Training Progress

The model showed consistent improvement across epochs:

Epoch	Training Loss	Validation Loss	Accuracy	F1 Score
1	0.2259	0.2567	92.89%	92.88%
2	0.2002	0.2360	93.44%	93.43%
3	0.1388	0.1925	96.11%	96.11%
4	0.1322	0.2161	95.67%	95.67%
5	0.1182	0.2208	96.22%	96.22%
6-10	0.1170-0.1171	0.2132-0.2142	95.67-95.78%	95.67-95.78%

Usage

Loading the Model

python

from transformers import ViTImageProcessor, ViTForImageClassification
from PIL import Image
import torch

# Load model and processor
model = ViTForImageClassification.from_pretrained("YOUR_USERNAME/vit-deepfake-detector")
processor = ViTImageProcessor.from_pretrained("YOUR_USERNAME/vit-deepfake-detector")

# Load and preprocess image
image = Image.open("path_to_image.jpg")
inputs = processor(images=image, return_tensors="pt")

# Make prediction
with torch.no_grad():
    outputs = model(**inputs)
    logits = outputs.logits
    predicted_class = logits.argmax(-1).item()

# Get label
labels = {0: "real", 1: "fake"}
print(f"Prediction: {labels[predicted_class]}")

# Get confidence scores
probabilities = torch.nn.functional.softmax(logits, dim=-1)
confidence = probabilities[0][predicted_class].item()
print(f"Confidence: {confidence:.2%}")

Batch Prediction

python

from transformers import pipeline

# Create classification pipeline
classifier = pipeline("image-classification", model="YOUR_USERNAME/vit-deepfake-detector")

# Predict on single image
result = classifier("path_to_image.jpg")
print(result)

# Predict on multiple images
images = ["image1.jpg", "image2.jpg", "image3.jpg"]
results = classifier(images)
for img, result in zip(images, results):
    print(f"{img}: {result}")

Limitations and Biases

Known Limitations

Dataset Bias: The model was trained on the OpenForensics dataset, which may not represent all types of deepfakes or manipulation techniques
Generalization: Performance may degrade on deepfake generation methods not present in the training data
Adversarial Robustness: The model has not been explicitly tested against adversarial attacks
Resolution Dependency: Best performance on images around 224x224 pixels (ViT input size)

Potential Biases

The model's performance may vary across different:
- Image sources and quality levels
- Demographic representations in images
- Types of manipulation techniques
- Content domains (faces, landscapes, objects, etc.)

Ethical Considerations

This model should be used responsibly and not for harassment or privacy invasion
Decisions based on this model should involve human oversight, especially in high-stakes scenarios
Users should be aware that deepfake detection is an evolving field, and no model is perfect
False positives and false negatives can have real-world consequences

Citation

If you use this model, please cite:

bibtex

@misc{vit-deepfake-detector,
  author = {YOUR_NAME},
  title = {ViT Deepfake Detection Model},
  year = {2024},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/YOUR_USERNAME/vit-deepfake-detector}}
}

Author

Dr. Lucy Liu
Muhammad Hamza Sohail
Ayaan Mohammed
Shadab Karim
kirti Dhir

Disclaimer:

his model is provided for research and educational purposes. Users are responsible for ensuring compliance with applicable laws and ethical guidelines when deploying this model.

Hamzenium/ViT-Deepfake-Classifier

ViT Deepfake Detection Model

Model Description

Model Details

Intended Use

Primary Use Cases

Out-of-Scope Use

Training Data

Training Procedure

Hyperparameters

Training Hardware

Data Augmentation

Performance

Validation Set Results (Best Epoch - Epoch 5)

Test Set Results

Training Progress

Usage

Loading the Model

Batch Prediction

Limitations and Biases

Known Limitations

Potential Biases

Ethical Considerations

Citation

Author

Disclaimer:

Hamzenium/
ViT-Deepfake-Classifier