You can deploy this model and start earning money today!

nwpuluka/
lovif-paper-submission-checkpoint

Name: lovif-paper-submission-checkpoint
Availability: InStock

multimodal-rating-prediction

mit

Views

No views yet

LoViF: LatentQuery Fusion Baseline Checkpoint (DistilBERT + DINOv2)

This repository contains the official baseline checkpoint for the LoViF 2026 Efficiency Track multimodal rating prediction task.

Model Description

The model implements a LatentQuery Fusion architecture, designed for high-efficiency multimodal inference. It fuses visual features from DINOv2 and textual features from DistilBERT using a cross-attention based latent query mechanism.

Vision Backbone: facebook/dinov2-base (LoRA adapter)
Text Backbone: distilbert/distilbert-base-uncased (LoRA adapter)
Fusion Method: LatentQuery Fusion (LQF)
Total Parameters: 186.09M
Trainable Parameters (LoRA): 33.15M

Performance Metrics

Based on our validation set (SetA validation split):

PLCC (Pearson Linear Correlation Coefficient): 0.2627
MSE (Mean Squared Error): 0.5823
FLOPs per sample (k=2): 116.79G
FLOPs per sample (k=1): 70.58G

Usage

To use this checkpoint, clone the LoViF repository and follow the reproduction steps in the README.

Download via CLI

huggingface-cli download nwpuluka/lovif-paper-submission-checkpoint best.ckpt --local-dir .

Loading in Python

python

import torch
checkpoint = torch.load("best.ckpt", map_location="cpu")
model_state = checkpoint["model_state"]
# Load model_state into your LoViF model instance

Reproduction

This checkpoint was trained using the baseline_lqf_distilbert_dinov2_train.yaml configuration for one round of optimization with modality dropout enabled.

Citation

If you use this model in your research, please cite our LoViF 2026 paper.