LLaVA-1.5-SDPO (Symmetric Polarity-Inverted DPO)
This model card describes the visual alignment model LLaVA-1.5-SDPO, a vision-language model (VLM) fine-tuned using a 4-term Symmetric Polarity-Inverted Preference Loss (SymDPO/SDPO) to enhance visual intelligence, specifically in comprehending and logically reasoning about optical and visual illusions.
Model Details
Model Description
Standard vision-language models frequently fail basic visual intelligence and spatial consistency tests. For instance, when presented with a visual illusion, their responses often change inconsistently based on how the question is framed.
This model is fine-tuned from LLaVA-1.5-7B on a custom polarity-inverted preference dataset. By applying Symmetric Polarity Direct Preference Optimization (SDPO), the model is trained to remain logically and visually consistent when prompt polarity is inverted (e.g. asking which element appears "longer" vs. "shorter") on the exact same static illusion image.
- Developed by: Matisse van Schalkwijk
- Model type: Vision-Language Model (LoRA Adapter on
llava-hf/llava-1.5-7b-hf language model + fine-tuned multi-modal projector)
- Language(s): English
- License: Apache 2.0 / LLaVA Research License
- Finetuned from model: llava-hf/llava-1.5-7b-hf
Model Sources
Uses
Direct Use
- Visual Intelligence Research: Psychometric evaluation and probing of VLMs on geometric (Müller-Lyer, Ponzo, Ebbinghaus), color/contrast (Simultaneous Contrast, White's Illusion), angle (Zöllner, Poggendorff), and motion (Scintillating Grid) visual illusions.
- Consistency Analysis: Studying spatial and semantic consistency under opposite prompt framings.
Out-of-Scope Use
- Critical decision-making applications (e.g., medical imaging analysis, autonomous driving visual perception, high-stakes safety tasks) where guaranteed visual accuracy is required without human-in-the-loop oversight.
Bias, Risks, and Limitations
Like all large vision-language models, LLaVA-1.5-SDPO is subject to hallucination, social biases inherent in its pretraining data, and varying accuracy across complex scenes. It is primarily intended as a research release for evaluating VLM consistency and visual intelligence.
How to Get Started with the Model
Because this fine-tuning run updates both the language backbone (via LoRA adapters) and the multimodal projector weights, you should load both components. Use the snippet below to download and initialize the model:
1import torch
2from transformers import pipeline, AutoProcessor
3from peft import PeftModel
4from huggingface_hub import hf_hub_download
5
6# 1. Initialize base LLaVA-1.5 model and processor
7model_id = "llava-hf/llava-1.5-7b-hf"
8adapter_id = "Matisse6410/LlaVa-1.5-SDPO"
9
10processor = AutoProcessor.from_pretrained(model_id)
11pipe = pipeline("image-text-to-text", model=model_id, torch_dtype=torch.bfloat16, device_map="auto")
12
13# 2. Load the trained LoRA adapter weights
14pipe.model = PeftModel.from_pretrained(pipe.model, adapter_id)
15
16# 3. Download and load the custom fine-tuned multi-modal projector weights
17projector_file = hf_hub_download(repo_id=adapter_id, filename="multi_modal_projector.pt")
18projector_state = torch.load(projector_file, map_location=pipe.model.device)
19pipe.model.base_model.model.model.multi_modal_projector.load_state_dict(projector_state)
20
21# Now ready for inference!
Training Details
Training Data
The model was trained on the Symmetric Polarity-Inverted Preference Dataset, consisting of:
- Polarity Pairs: Textual prompts and corresponding chosen/rejected response pairs representing visual illusions across categories: Geometric, Color, Angle, and Motion.
- Control VQA Safeguard: Approximately 20% of the training data consists of non-illusion factual control visual question-answering entries (e.g., "How many lines are in this image?", "What colour is the background?") to mitigate catastrophic forgetting of general visual capabilities during the preference alignment process.
Training Procedure
Fine-tuning is performed using the Symmetric Polarity Preference Loss formulation:
$$\mathcal{L}(\theta) = \mathcal{L}{\text{DPO}, m}(\theta) + \gamma \mathcal{L}{\text{Symmetric}}(\theta) + \lambda \mathcal{L}{\text{Margin}}(\theta) + \eta \mathcal{L}{\text{AncPO}}(\theta)$$
This multi-term loss function optimizes:
- Standard DPO Loss on the original prompt polarity.
- Symmetric DPO Loss on the inverted prompt polarity.
- Preference-Margin Consistency Loss to minimize variance between the original and inverted preference gaps.
- Anchored Preference Loss (AncPO) to stabilize the absolute log-likelihoods of chosen responses.
During training, the CLIP vision encoder remains frozen, the multi-modal projector is fully unfrozen and updated, and the language model is adapted using LoRA on its projection layers.
Training Hyperparameters
- DPO Temperature ($\beta$): 0.1
- Symmetric Loss Weight ($\gamma$): 1.0
- Preference Margin Weight ($\lambda$): 0.5
- Anchored Preference Weight ($\eta$): 0.1
- LoRA Rank ($r$): 64
- LoRA Alpha ($\alpha$): 16
- LoRA Dropout: 0.05
- Learning Rate: $1.0 \times 10^{-5}$
- Learning Rate Schedule: Linear warmup (first 5% of steps) followed by Cosine learning rate decay.
- Optimizer: AdamW
Environmental Impact
- Hardware Type: NVIDIA GPUs (A100 / H100 cluster)
- Precision: BF16 Mixed Precision
Model Card Authors
Framework Versions
- PEFT 0.19.1
- PyTorch 2.4+
- Transformers 4.45+
