You can deploy this model and start earning money today!

mlx-community/
sam3-image

Name: sam3-image
Availability: InStock

Views

No views yet

🎯 MLX SAM3

Segment Anything Model 3 — Native Apple Silicon Implementation

This is an MLX port of Meta's SAM3 model, optimized for native execution on Apple Silicon (M1/M2/M3/M4) Macs.

📖 Learn more: Check out the accompanying blog post explaining the SAM3 architecture and this implementation.

Model Description

SAM3 (Segment Anything Model 3) is a powerful image segmentation model that can segment objects in images using:

Text prompts — Describe what you want to segment ("car", "person", "dog")
Box prompts — Draw bounding boxes to include or exclude regions

This MLX port provides native Apple Silicon performance, leveraging Apple's MLX framework for optimized inference on Mac.

Intended Uses

Interactive image segmentation on Apple Silicon Macs
Object detection and masking with text descriptions
Region-based segmentation using bounding box prompts
Rapid prototyping of segmentation workflows on Mac

How to Use

Installation

bash

# Clone the repository
git clone https://github.com/Deekshith-Dade/mlx_sam3.git
cd mlx-sam3

# Install with uv (recommended)
uv sync

# Or with pip
pip install -e .

Python API

python

from PIL import Image
from sam3 import build_sam3_image_model
from sam3.model.sam3_image_processor import Sam3Processor

# Load model (auto-downloads weights on first run)
model = build_sam3_image_model()
processor = Sam3Processor(model, confidence_threshold=0.5)

# Load and process an image
image = Image.open("your_image.jpg")
state = processor.set_image(image)

# Segment with text prompt
state = processor.set_text_prompt("person", state)

# Access results
masks = state["masks"]       # Binary segmentation masks
boxes = state["boxes"]       # Bounding boxes [x0, y0, x1, y1]
scores = state["scores"]     # Confidence scores

print(f"Found {len(scores)} objects")

Web Interface

Launch the interactive web application:

cd app && ./run.sh

Frontend: http://localhost:3000
API: http://localhost:8000
API Docs: http://localhost:8000/docs

Requirements

Requirement	Version	Notes
macOS	13.0+	Apple Silicon required (M1/M2/M3/M4)
Python	3.13+	Required for MLX compatibility
Node.js	18+	For the web interface (optional)

⚠️ Apple Silicon Only: This implementation uses MLX, which is optimized exclusively for Apple Silicon.

Model Details

Architecture: SAM3 with ViTDet backbone
Framework: MLX (Apple's machine learning framework)
Weights: Converted from original PyTorch weights
Model Size: ~3.5GB

Limitations

Runs only on Apple Silicon Macs (M1/M2/M3/M4)
Requires macOS 13.0 or later
Python 3.13+ required for MLX compatibility

Citation

If you use this model, please cite the original SAM3 paper and this MLX implementation:

bibtex

@misc{mlx-sam3,
  author = {Deekshith Dade},
  title = {MLX SAM3: Native Apple Silicon Implementation},
  year = {2024},
  url = {https://github.com/Deekshith-Dade/mlx_sam3}
}

Acknowledgments

Meta AI for the original SAM3 model
Apple MLX Team for the MLX framework
The open-source community for continuous inspiration

Built with ❤️ for Apple Silicon

mlx-community/sam3-image