AffKernel: IIT-AFF affordance segmentation (R50vd, stride-2, full model)
Released weights for AffKernel: single-pass, NMS-free affordance
segmentation. RT-DETR object queries generate per-query dynamic convolution
kernels that decode per-object affordance masks from one shared high-resolution
map.
This card describes the artifacts only. Method, results, ablations,
deployment benchmarks, protocol caveats, licensing detail and citation all
live in the code repository and are deliberately not repeated here:
https://github.com/anh0001/affkernel
(reproduction guide,
dataset setup)
What this checkpoint is
IIT-AFF, 72 epochs, seed 42, last-epoch EMA weights, fp32.
F_beta^w (beta^2 = 1)
0.8685
F_beta^w (beta^2 = 0.3)
0.8582
This is one seed, not the headline mean: the three-seed figure is
0.8675 ± 0.0009, and seed 42 is the highest of the three (7: 0.8668,
123: 0.8673). Both beta conventions are given because a beta^2 = 1 number
must never be compared against a beta^2 = 0.3 one. Read the accuracy as
exploratory rather than confirmatory: IIT-AFF ships no validation split, so the
same test set also informed the recipe. Full reasoning in the repository README.
Inference, evaluation and deployment commands are in the
repository README.
Files
File
What it is
affkernel_iit_r50vd_stride2_deepsup_seed42.pth
The model. fp32 EMA weights, 174 MB.
backbone_fp16.plan
Optional TensorRT fp16 backbone-only engine, 46.4 MiB. Not portable, not usable on its own.
deepsup in the filename is the historical identifier for the auxiliary readout
losses used during training; it is kept so the documented commands keep working.
Checkpoint format
A single file holding the EMA weights only:
{"ema": {"module": <OrderedDict of 760 tensors>, "updates": 111312}}
The evaluation path reads the EMA parameters, so raw weights and optimizer state
are not included. Do not repack this into a bare {"model": ...} dict: under
an EMA-enabled config the solver would then evaluate a freshly initialised EMA
module and score near zero.
The first run fetches the ImageNet-pretrained ResNet-50vd backbone from the
RT-DETR release artefacts, so it needs network access. Set
PResNet.pretrained: False for an offline deployment.
backbone_fp16.plan
The dynamic-kernel affordance head cannot be exported to ONNX, but the backbone
is plain convolutions and is the largest single term of the forward pass
(17.7 ms to 5.3 ms). The encoder, decoder and affordance head stay in PyTorch,
CUDA-graphed against the engine's output buffers. Built on Jetson AGX Orin
(64 GB), JetPack 6.2 (L4T R36.4.7), TensorRT 10.3.0 / CUDA 12.6; input images(1, 3, 640, 640) fp16 fixed; outputs feat0..feat3 fp16.
Engine files are not portable. A .plan is compiled for one specific GPU,
TensorRT version and input shape. On any other device, JetPack/TensorRT version
or input size it will fail to deserialize, or load and behave incorrectly. It
is published here only as a convenience for setups identical to the above.
If yours differs in any respect, build your own — one command, a few
minutes, and it is the supported path. See
Deployment.
Intended use and limitations
Research on affordance perception and perception-guided grasping.
Closed vocabulary: 10 object and 9 affordance classes, listed in
docs/datasets.md.
It will not generalise to unseen categories.
IIT-AFF is a tabletop dataset. Other viewpoints, lighting or clutter regimes
are untested.
The residual error is dominated by missed detections rather than by mask
quality.
Not validated for safety-critical deployment. A predicted grasp region is a
perception cue, not a guarantee of a safe grasp.
License
Weights are Apache-2.0, matching the RT-DETR components and the
ImageNet-pretrained backbone they were initialised from. The IIT-AFF dataset
is not redistributed here; obtain it from
https://sites.google.com/site/iitaffdataset/. Full attribution and the citation
BibTeX are in the
repository.
