This model was supervised-trained on real and fake news pairs to better detect misinformation in South Asian digital content. During inference, it uses prompt-based similarity to classify inputs.

Deployed at:
🔗 https://huggingface.co/jarif/Multimodal-BNEN-Fake-News-Scanner-Model

✅ Key Features

✅ Supervised fine-tuning on fake/real news dataset
📝 Bangla + English text support
🖼️ Analyzes image authenticity
🔍 Uses prompt-based classification with semantic similarity
🧩 Built on openai/clip-vit-base-patch32, fine-tuned for misinformation detection
🌍 Optimized for South Asian context
📦 Fully compatible with Hugging Face transformers

💻 Inference Code

Classify images and text as Real or Fake and display results in a clean table using tabulate.

Install Dependencies

python

pip install transformers torch pillow tabulate

from transformers import CLIPModel, CLIPProcessor
from PIL import Image
import torch
import torch.nn.functional as F
from tabulate import tabulate

# Load your fine-tuned model
model = CLIPModel.from_pretrained("jarif/Multimodal-BNEN-Fake-News-Scanner-Model")
processor = CLIPProcessor.from_pretrained("jarif/Multimodal-BNEN-Fake-News-Scanner-Model")

# Define class prompts in Bangla
class_texts = ["এটি ফেক নিউজ", "এটি রিয়েল নিউজ"]  # ["This is fake news", "This is real news"]

# --- Image Classification ---
image = Image.open("your_image.jpg").convert("RGB")  # Replace with your image path
image_inputs = processor(images=image, return_tensors="pt")
image_emb = model.get_image_features(**image_inputs)

# --- Text Classification ---
text = "পদ্মা নদীর প্রবল স্রোতে লঞ্চঘাট বিলীন হয়েছে।"
text_inputs = processor(text=text, return_tensors="pt", padding=True, truncation=True)
text_emb = model.get_text_features(**text_inputs)

# Get embeddings for class prompts
class_inputs = processor(text=class_texts, return_tensors="pt", padding=True, truncation=True)
class_embs = model.get_text_features(**class_inputs)

# Normalize embeddings (cosine similarity)
image_emb = F.normalize(image_emb, p=2, dim=-1)
text_emb = F.normalize(text_emb, p=2, dim=-1)
class_embs = F.normalize(class_embs, p=2, dim=-1)

# Compute similarity
image_sims = (image_emb @ class_embs.T).squeeze(0)
text_sims = (text_emb @ class_embs.T).squeeze(0)

# Predict
image_pred = image_sims.argmax().item()
text_pred = text_sims.argmax().item()

image_label = "🛑 Fake" if image_pred == 0 else "✅ Real"
text_label = "🛑 Fake" if text_pred == 0 else "✅ Real"

# Create result table
table = [
    ["ImageRelation", image_label],
    ["Text Relation", text_label]
]

# Print formatted table
print(tabulate(table, headers=["Modality", "Prediction"], tablefmt="fancy_grid"))

📊 Example Output

╒════════════════════════════════════════════════════════════════════╕
│                         Modality                │ Prediction │
╞════════════════════════════════════════════════════════════════════╡
│ Image Relation                                  │ ✅ Real    │
│ Text Relation                                   │ 🛑 Fake    │
╘════════════════════════════════════════════════════════════════════╛

📎 License

This model is licensed under Apache 2.0.
Fine-tuned by Sadik Al Jarif for public misinformation detection.

🙌 Acknowledgements

OpenAI CLIP
Hugging Face Transformers
Bangla NLP Community