Self-contained Hugging Face checkpoints converted from the official DeCUR pretrained weights. Each subfolder ships remote code (modeling_decur.py, processor, pipeline) and loads with trust_remote_code=True.

Original paper: DeCUR: Decoupling Common and Unique Representations for Multimodal Self-Supervision
Converted for Hugging Face by: BiliSakura

Available checkpoints (16 models)

Folder	Backbone	Modality	Channels	RDA
`decur-resnet50-s1`	ResNet-50	Sentinel-1 SAR	2	no
`decur-resnet50-s2c`	ResNet-50	Sentinel-2 L1C	13	no
`decur-resnet50-rda-s1`	ResNet-50 + RDA	Sentinel-1 SAR	2	yes
`decur-resnet50-rda-s2c`	ResNet-50 + RDA	Sentinel-2 L1C	13	yes
`decur-vit-small-patch16-s1`	ViT-S/16	Sentinel-1 SAR	2	no
`decur-vit-small-patch16-s2c`	ViT-S/16	Sentinel-2 L1C	13	no
`decur-resnet50-rgb`	ResNet-50	GeoNRW RGB	3	no
`decur-resnet50-dem`	ResNet-50	GeoNRW DEM	3	no
`decur-resnet50-rda-rgb`	ResNet-50 + RDA	GeoNRW RGB	3	yes
`decur-resnet50-rda-dem`	ResNet-50 + RDA	GeoNRW DEM	3	yes
`decur-vit-small-patch16-rgb`	ViT-S/16	GeoNRW RGB	3	no
`decur-vit-small-patch16-dem`	ViT-S/16	GeoNRW DEM	3	no
`decur-mit-b2-rgb`	SegFormer MiT-B2	SUN RGB-D RGB	3	no
`decur-mit-b2-hha`	SegFormer MiT-B2	SUN RGB-D HHA	3	no
`decur-mit-b5-rgb`	SegFormer MiT-B5	SUN RGB-D RGB	3	no
`decur-mit-b5-hha`	SegFormer MiT-B5	SUN RGB-D HHA	3	no

Legacy .pth filename mapping is in conversion_manifest.json.

Usage

Processors default to do_resize: false — pass images at native (H, W, C); only value rescaling (/255) is applied unless you enable normalization.

python

from transformers import pipeline
import numpy as np

REPO = "/path/to/DECUR-transformers"
SUBFOLDER = "decur-resnet50-s2c"

pipe = pipeline(
    task="decur-feature-extraction",
    model=REPO,
    trust_remote_code=True,
    model_kwargs={"subfolder": SUBFOLDER},
)

# Native Sentinel-2 patch (e.g. 512×512, 13 bands)
image = np.random.randint(0, 255, (512, 512, 13), dtype=np.uint8)
features = pipe(image, pool=True, return_tensors=True)
print(features.shape)  # torch.Size([1, 2048])

# Dense token map (spatial grid scales with input size)
tokens = pipe(image, pool=False, return_tensors=True)
print(tokens.shape)    # ResNet: [1, 256, 2048] for 512×512

Sentinel-1 ViT (2 channels):

python

SUBFOLDER = "decur-vit-small-patch16-s1"
pipe = pipeline(
    task="decur-feature-extraction",
    model=REPO,
    trust_remote_code=True,
    model_kwargs={"subfolder": SUBFOLDER},
)
image = np.random.randint(0, 255, (448, 448, 2), dtype=np.uint8)
features = pipe(image, pool=True, return_tensors=True)
print(features.shape)  # torch.Size([1, 384])

Load components directly:

python

from transformers import AutoModel, AutoImageProcessor

model = AutoModel.from_pretrained(REPO, subfolder=SUBFOLDER, trust_remote_code=True)
processor = AutoImageProcessor.from_pretrained(REPO, subfolder=SUBFOLDER, trust_remote_code=True)
inputs = processor(image, return_tensors="pt")
out = model(**inputs)

Each checkpoint folder can also be used as a standalone local path (omit subfolder):

python

pipe = pipeline(
    task="decur-feature-extraction",
    model="/path/to/DECUR-transformers/decur-mit-b2-rgb",
    trust_remote_code=True,
)

Native resolution notes

Backbone	Native larger input	Pooled output	Sequence output
ResNet-50	Yes (fully conv)	`[B, 2048]`	`[B, H'×W', 2048]`
ViT-S/16	Yes (`interpolate_pos_encoding=True` by default)	`[B, 384]` (CLS)	`[B, N, 384]`
MiT-B2/B5	Yes	`[B, 512]` (spatial mean)	`[B, H'×W', 512]`

size in preprocessor_config.json (224×224) is the pretraining reference, not a forced input size.
ViT uses interpolated positional embeddings by default for non-224 inputs. Disable with model(..., interpolate_pos_encoding=False).
Opt in to resize (e.g. match 224×224 pretraining):

python

features = pipe(
    image,
    pool=True,
    return_tensors=True,
    image_processor_kwargs={"do_resize": True},
)

The custom pipeline is registered in each checkpoint's config.json under custom_pipelines.decur-feature-extraction (see Adding a new pipeline).

Test CLI

bash

conda activate rsgen
python test_decur.py --model decur-mit-b2-rgb
python test_decur.py --all

Dependencies

transformers
torch
torchvision (ResNet backbones)
einops (RDA modules)
opencv-python (only when resizing multispectral inputs with >4 channels)

Notes

Pooled features: ResNet [B, 2048], ViT [B, 384] (CLS token), MiT [B, 512] (spatial mean).
GeoNRW DEM checkpoints use 3 input channels in the released weights (not 1-channel DSM).
RDA ResNet models include deformable attention modules (da_l3, da_l4).

BiliSakura/DECUR-transformers

DeCUR Transformers Models

Available checkpoints (16 models)

Usage

Native resolution notes

Test CLI

Dependencies

Notes

BiliSakura/
DECUR-transformers