You can deploy this model and start earning money today!

kerasformers/
maskformer-swin-tiny-coco

Name: maskformer-swin-tiny-coco
Availability: InStock

kerasformers

keras

maskformer

universal-segmentation

facebook/maskformer-swin-tiny-coco

finetune

cc-by-nc-4.0

Views

No views yet

See our collection for all versions of MaskFormer.

Run MaskFormer with Keras 3: JAX, PyTorch, or TensorFlow

GitHub

zeromodels/maskformer-swin-tiny-coco

Paper: Per-Pixel Classification is Not All You Need for Semantic Segmentation (arXiv:2107.06278) · HF Papers

MaskFormer reframes segmentation as mask classification: a backbone and pixel decoder feed a transformer decoder whose queries each predict a binary mask and a class. One architecture covers semantic, instance, and panoptic outputs via post-processing.

For more details on the model, please go to the upstream model card.

Pure-Keras 3 conversion of facebook/maskformer-swin-tiny-coco for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a universal segmentation checkpoint (MaskFormerUniversalSegment) trained on COCO panoptic.

✨ Quick start

python

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from zeromodels.models.maskformer import MaskFormerUniversalSegment, MaskFormerImageProcessor

model = MaskFormerUniversalSegment.from_weights("zeromodels/maskformer-swin-tiny-coco")
processor = MaskFormerImageProcessor.from_weights("zeromodels/maskformer-swin-tiny-coco")

image = Image.open("your_image.jpg").convert("RGB")
output = model(processor(image)["pixel_values"], training=False)
result = processor.post_process_panoptic_segmentation(
    output, target_size=(image.height, image.width)
)
print(result["segmentation"].shape)

Load any MaskFormer variant the same way with from_weights("zeromodels/<variant>"):

Variant	Hub	Dataset
`maskformer-swin-tiny-coco`	`zeromodels/maskformer-swin-tiny-coco`	COCO
`maskformer-swin-small-coco`	`zeromodels/maskformer-swin-small-coco`	COCO
`maskformer-swin-base-coco`	`zeromodels/maskformer-swin-base-coco`	COCO
`maskformer-swin-tiny-ade`	`zeromodels/maskformer-swin-tiny-ade`	ADE20K
`maskformer-swin-base-ade`	`zeromodels/maskformer-swin-base-ade`	ADE20K

Tips

Set KERAS_BACKEND before importing Keras / zeromodels.
Prefer MaskFormerImageProcessor.from_weights(...) so resolution matches the variant.
See MaskFormer docs and Loading Weights.
Community / upstream weights: MaskFormerUniversalSegment.from_weights("hf:facebook/maskformer-swin-tiny-coco").

Special Thanks

A huge thank you to the Facebook AI Research MaskFormer authors for creating and releasing these models.

License: CC-BY-NC-4.0 (non-commercial).

kerasformers/maskformer-swin-tiny-coco