You can deploy this model and start earning money today!

kerasformers/
detr-resnet-101-panoptic

Name: detr-resnet-101-panoptic
Availability: InStock

kerasformers

keras

detr

panoptic-segmentation

facebook/detr-resnet-101-panoptic

finetune

apache-2.0

Views

No views yet

See our collection for all versions of DETR.

Run DETR with Keras 3: JAX, PyTorch, or TensorFlow

GitHub

zeromodels/detr-resnet-101-panoptic

Paper: End-to-End Object Detection with Transformers (arXiv:2005.12872) · HF Papers

DETR (DEtection TRansformer) treats object detection as direct set prediction. A ResNet backbone produces a feature map, a transformer encoder-decoder attends over it with a fixed set of learned object queries, and each query emits one class and one box. Training uses a bipartite (Hungarian) matching loss, so every ground-truth object is assigned exactly one query. That framing removes anchors and NMS. Panoptic checkpoints add a mask head for things and stuff.

For more details on the model, please go to Facebook's original model card.

Pure-Keras 3 conversion of facebook/detr-resnet-101-panoptic for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a panoptic segmentation checkpoint (DETRPanopticSegment): each query predicts a class, box, and mask (COCO things + stuff).

✨ Quick start

python

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from zeromodels.models.detr import DETRPanopticSegment, DETRImageProcessor

model = DETRPanopticSegment.from_weights("zeromodels/detr-resnet-101-panoptic")
processor = DETRImageProcessor.from_weights("zeromodels/detr-resnet-101-panoptic")

image = Image.open("your_image.jpg").convert("RGB")
output = model(processor(image)["pixel_values"], training=False)
# output['logits'], output['pred_boxes'], output['pred_masks']

Load any DETR variant the same way with from_weights("zeromodels/<variant>") (use DETRPanopticSegment for this repo):

Variant	Hub	Task
`detr-resnet-50`	`zeromodels/detr-resnet-50`	object detection
`detr-resnet-101`	`zeromodels/detr-resnet-101`	object detection
`detr-resnet-50-panoptic`	`zeromodels/detr-resnet-50-panoptic`	panoptic segmentation
`detr-resnet-101-panoptic`	`zeromodels/detr-resnet-101-panoptic`	panoptic segmentation

Tips

Set KERAS_BACKEND before importing Keras / zeromodels.
Detection: DETRDetect + post_process_object_detection (try threshold=0.9 on clean COCO scenes).
Panoptic: DETRPanopticSegment returns pred_masks per query; threshold at zero and resize to the image size yourself.
See DETR docs and Loading Weights.
Community / upstream safetensors still work via the hf: prefix, e.g. DETRPanopticSegment.from_weights("hf:facebook/detr-resnet-101-panoptic").

Special Thanks

A huge thank you to the Facebook AI Research DETR authors for creating and releasing these models.

License: Apache 2.0.

kerasformers/detr-resnet-101-panoptic