Multi-Scale Efficient Global Context Vision Transformer (MS-EffGCViT) is a hybrid CNN-ViT
architecture for deepfake detection. It fuses CNN-driven spatial inductive bias with
hierarchical global-context attention to catch both local artifacts (textures, blending seams)
and global artifacts (lighting, structural inconsistency).
A single architecture ships in two sizes and three domain-tuned checkpoints, working on both
static images and video at the frame level.
Learning to detect manipulated facial images
[Paper] [Download],
featuring
1,000 original YouTube videos manipulated by 5 face forgery methods.
Trained and tested on the same dataset.
1pip install deepguard
2from transformers import pipeline
1clf = pipeline(
2 "image-classification",
3 model="KoreaPeter/ms-eff-gcvit-deepfake-b5-ff-plus-plus",
4 trust_remote_code=True,
5)
6
7# ── Basic Inference ───────────────────────────────────────────────
8result = clf("face.jpg")
9# [{'label': 'fake', 'score': 0.9712}, {'label': 'real', 'score': 0.0288}]
10
11# ── Custom Parameters ─────────────────────────────────────────────
12result = clf(
13 "face.jpg",
14 margin_ratio=0.2, # Margin ratio around the detected face bbox (default: 0.2)
15 conf_thres=0.5, # Confidence threshold for YOLO face detection (default: 0.5)
16 min_face_ratio=0.01, # Minimum face-to-frame area ratio to process (default: 0.01)
17 tta_hflip=0.0, # Probability of horizontal flip for TTA (default: 0.0)
18 top_k=1, # Number of top labels to return (default: all)
19)
20# [{'label': 'fake', 'score': 0.9712}]
1clf = pipeline(
2 "video-classification",
3 model="KoreaPeter/ms-eff-gcvit-deepfake-b5-ff-plus-plus",
4 trust_remote_code=True,
5)
6
7# ── Basic Inference ───────────────────────────────────────────────
8result = clf("video.mp4")
9# [{'label': 'fake', 'score': 0.9634}, {'label': 'real', 'score': 0.0366}]
10
11# ── Custom Parameters ─────────────────────────────────────────────
12result = clf(
13 "video.mp4",
14 num_frames=20, # Number of frames to sample (default: 20)
15 margin_ratio=0.2, # Margin ratio around the detected face bbox (default: 0.2)
16 conf_thres=0.5, # Confidence threshold for YOLO face detection (default: 0.5)
17 min_face_ratio=0.01, # Minimum face-to-frame area ratio to process (default: 0.01)
18 tta_hflip=0.0, # Probability of horizontal flip for TTA (default: 0.0)
19 agg_mode="conf", # Aggregation mode: 'conf' | 'mean' | 'vote' (default: 'conf')
20 return_frame_scores=True, # Return per-frame scores (default: False)
21)
22# [{'label': 'fake', 'score': 0.9634},
23# {'label': 'real', 'score': 0.0366},
24# {'frame_scores': [0.97, 0.95, 0.98, ...], 'agg_mode': 'conf'}]
While traditional Vision Transformers (ViTs) utilize a Linear Projection for patch embedding, our proposed model adopts a CNN-based Patch Embedding module incorporating MBConvBlocks.
We utilizes two distinct types of self-attention to capture both long-range and short-range information across feature maps.
-
Local Window Attention: this model efficiently captures local textures and precise spatial details while maintaining linear computational complexity relative to the image size.
-
Global Window Attention: Unlike Swin Transformer, this module utilizes global-queries that interact with local window keys and values. This allows each local region to incorporate global context, effectively capturing long-range dependencies and providing a comprehensive understanding of the entire spatial structure
-
Efficient Backbone
While both Xception and EfficientNet show great results on DeepFake benchmarks, EfficientNet is chosen for its superior computational efficiency. By utilizing MBconv (Inverted Residual Blocks) and depthwise convolutions, it achieves significantly lower FLOPS compared to Xception.
-
Window-based Attention: Instead of applying self-attention on raw images, this model operates on feature maps extracted from backbone blocks. By partitioning these maps into windows, the $O(N^2)$ complexity is restricted to the window size, siginificantly lowering the computational footprint.
Modern DeepFakes can leave very localized forgery region. To Capture this, we adopts a multi-scale strategy by extracting features from different levels of the backbone.
-
(Subtle Artifacts): High-Resolution feature maps are extracted from early backbone blocks(
l_block_idx) to capture like skin texture or boundary artifacts
-
(Global Features): Low-Resolution feature maps are extracted from deeper blocks(
h_block_idx) to analyze overall lighting, shadows, and structural consistency.
-
Feature Fusion: The Outputs from both branches (L-GCViT and H-GCViT) are fused to make a comprehensive decision based on both local and global context.
1@misc{deepguard2026,
2 title = {DeepGuard: Multi-Scale Efficient Global Context Vision Transformer for Deepfake Detection},
3 author = {seoyunje},
4 year = {2026},
5 url = {https://github.com/HanMoonSub/DeepGuard}
6}
-00C853?style=flat-square){kind=icon}
