CLIP ViT-B/32 — Multimodal Misinformation Detection on FakeTT
Authors: Andrei-Gabriel Radu, Ciprian-Octavian Truică, Elena-Simona Apostol
National University of Science and Technology POLITEHNICA Bucharest
Supervised multimodal misinformation classification checkpoint based on CLIP ViT-B/32, trained and evaluated on FakeTT.
This model accompanies the bachelor thesis Misinformation Detection in Social Media Videos.
Results
| Dataset | Modality | Macro-F1 |
|---|
| FakeTT | Multimodal | 0.8748 |
Model
- Architecture: CLIP ViT-B/32
- Task: Binary misinformation classification
- Modality: Text + video
- Dataset: FakeTT
- Number of classes: 2
- Checkpoint:
pytorch_model.bin
- Primary metric: Macro-F1
Saved Metadata
- batch_size: 8
- dropout: 0.2
- early_stopping_patience: 3
- freeze_backbone: False
- fusion_hidden_dim: 256
- gradient_accumulation_steps: 1
- learning_rate: 1e-05
- lr_scheduler_type: linear
- max_grad_norm: 1.0
- model_name: openai/clip-vit-base-patch32
- num_labels: 2
- num_train_epochs: 30
- text_max_length: 77
- warmup_ratio: 0.0
- weight_decay: 0.01
The original machine-readable metadata is included as model_metadata.json.
Usage
This repository contains a checkpoint for the custom multimodal classifier used in the thesis implementation. It is not a drop-in AutoModel.from_pretrained() repository.
Instantiate the matching custom classifier, load the included tokenizer and image/video processor, then load the state dictionary:
1import torch
2
3state = torch.load("pytorch_model.bin", map_location="cpu")
4
5# model = MatchingClassifier(...)
6# model.load_state_dict(state)
7# model.eval()
The constructor and preprocessing must match the thesis implementation and model_metadata.json.
Intended Use
Research, benchmarking and reproducibility of supervised multimodal misinformation detection on short social-media videos.
Limitations
The model performs classification rather than factual verification, can inherit biases from FakeTT and its pretrained encoders, and requires the matching custom implementation for reproducible inference.
Citation
1@thesis{radu2026misinformation,
2 author = {Radu, Andrei-Gabriel and Truică, Ciprian-Octavian and Apostol, Elena-Simona},
3 title = {Misinformation Detection in Social Media Videos},
4 school = {National University of Science and Technology POLITEHNICA Bucharest},
5 year = {2026}
6}
