Views
No views yet
pip install -U mlx-vlm1from pathlib import Path
2from PIL import Image
3from mlx_vlm.utils import load_model
4from mlx_vlm.models.rfdetr.processing_rfdetr import RFDETRProcessor
5from mlx_vlm.models.rfdetr.generate import RFDETRPredictor
6
7model = load_model(Path("mlx-community/rfdetr-seg-large-fp32"))
8processor = RFDETRProcessor.from_pretrained("mlx-community/rfdetr-seg-large-fp32")
9predictor = RFDETRPredictor(model, processor, score_threshold=0.3, nms_threshold=0.5)
10
11result = predictor.predict(Image.open("image.jpg"))
12# result.boxes - (N, 4) xyxy pixel coordinates
13# result.scores - (N,) confidence scores
14# result.masks - (N, H, W) binary instance masks
15# result.class_names - list of class names1# Image segmentation
2python -m mlx_vlm.models.rfdetr.generate --task segment --image photo.jpg --model mlx-community/rfdetr-seg-large-fp32
3
4# Video processing
5python -m mlx_vlm.models.rfdetr.generate --task track --video input.mp4 --model mlx-community/rfdetr-seg-large-fp32
6
7# Realtime camera
8python -m mlx_vlm.models.rfdetr.generate --task realtime --model mlx-community/rfdetr-seg-large-fp32
9
10# Blur background
11python -m mlx_vlm.models.rfdetr.generate --task segment --image photo.jpg --model mlx-community/rfdetr-seg-large-fp32 --annotator blur+bg| Architecture | DINOv2-small backbone + C2f projector + 5-layer Deformable DETR decoder + 5-block Segmentation head |
| Task | Object detection + instance segmentation (COCO 80 classes) |
| Parameters | ~36M |
| Input resolution | 504x504 |
| Mask resolution | 126x126 |
| Dtype | float32 |
| Inference (M4 Max) | ~89ms per image (~11 FPS) |
| Peak memory | ~1.2 GB |
| Model | Resolution | Mask quality | Inference | Memory |
|---|---|---|---|---|
| rfdetr-seg-small-fp32 | 384px (96x96 masks) | Good | ~26ms (38 FPS) | 683 MB |
| rfdetr-seg-large-fp32 | 504px (126x126 masks) | Better | ~89ms (11 FPS) | 1.2 GB |