# Install dependencies
pip install torch torchvision torchaudio
pip install transformers peft accelerate
pip install qwen-vl-utils imageio[ffmpeg]

# Run inference
python inference_mac.py your_video.mp4

Example Code

python

import torch
from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
from peft import PeftModel

# Use MPS for Apple Silicon
device = "mps" if torch.backends.mps.is_available() else "cpu"

# Load model
model = Qwen2VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2-VL-2B-Instruct",
    torch_dtype=torch.float16
).to(device)

model = PeftModel.from_pretrained(model, "Raullen/petvlm")
model.eval()

processor = AutoProcessor.from_pretrained("Raullen/petvlm")

For complete Mac installation guide and optimized inference script, see:

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 1
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 4
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.05
num_epochs: 15.0

Training results

Training Loss	Epoch	Step	Validation Loss
3.2859	0.6369	25	2.8292
2.0143	1.2548	50	1.9940
1.6829	1.8917	75	1.7856
1.4774	2.5096	100	1.7516
1.1908	3.1274	125	1.6817
1.2046	3.7643	150	1.6712
0.8012	4.3822	175	1.7618
0.9595	5.0	200	1.7216
0.5115	5.6369	225	1.8212
0.4666	6.2548	250	2.1777
0.4082	6.8917	275	2.0293
0.2262	7.5096	300	2.3396
0.1774	8.1274	325	2.3459
0.1512	8.7643	350	2.4745
0.0719	9.3822	375	2.7719
0.0741	10.0	400	2.7706
0.0679	10.6369	425	2.8739
0.036	11.2548	450	3.0192
0.03	11.8917	475	3.0430
0.0226	12.5096	500	3.1513
0.0155	13.1274	525	3.1610
0.015	13.7643	550	3.1778
0.0169	14.3822	575	3.2098
0.0223	15.0	600	3.2078

Framework versions

PEFT 0.17.1
Transformers 4.57.1
Pytorch 2.10.0+cu128
Datasets 4.0.0
Tokenizers 0.22.2