You can deploy this model and start earning money today!

BiliSakura/
FiT-diffusers

unconditional-image-generation

2402.12376

2410.13925

apache-2.0

Views

No views yet

FiT-diffusers

Diffusers-ready checkpoints for Flexible Vision Transformer (FiT) and FiTv2, converted from InfImagine/FiT / InfImagine/FiTv2.

Re-distribution notice: weights and configs in this repo are re-packaged from the official InfImagine releases. Original work: FiT (arXiv:2402.12376), FiTv2 (arXiv:2410.13925). License: Apache 2.0.

This repo is derived from the development bundle in Visual-Generative-Foundation-Model-Collection. Inference only needs:

This model repo (BiliSakura/FiT-diffusers)
PyPI diffusers, torch, safetensors

Each variant subfolder is a self-contained Diffusers model repo with:

model_index.json (includes ImageNet id2label)
pipeline.py (FiTPipeline for FiTv1, FiTv2Pipeline for FiTv2)
transformer/fit_transformer_2d.py and weights
scheduler/scheduler_config.json
vae/diffusion_pytorch_model.safetensors (stabilityai/sd-vae-ft-ema)

Demo

Class-conditional sample (ImageNet class 207, golden retriever), FiTv1-XL/2 at 256×256, 250 steps, CFG 1.5, seed 42.

Available checkpoints

Checkpoint	Path	Resolution	Sampler	Steps	CFG	FID (native res)
FiTv1-XL/2	`FiTv1-XL-2-256/`	256×256	improved diffusion (DDPM respaced)	250	1.5	4.21
FiTv2-XL/2	`FiTv2-XL-2-256/`	256×256	flow matching (velocity ODE)	250	1.5	2.26
FiTv2-3B/2	`FiTv2-3B-2-256/`	256×256	flow matching (velocity ODE)	250	1.5	2.15
FiTv2-HR-XL/2	`FiTv2-XL-2-512/`	512×512	flow matching (velocity ODE)	250	1.5	2.90
FiTv2-HR-3B/2	`FiTv2-3B-2-512/`	512×512	flow matching (velocity ODE)	250	1.5	2.41

Inference

python

from pathlib import Path
import torch
from diffusers import DiffusionPipeline

model_dir = Path("./FiTv2-XL-2-256")
pipe = DiffusionPipeline.from_pretrained(
    str(model_dir),
    local_files_only=True,
    custom_pipeline=str(model_dir / "pipeline.py"),
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
).to("cuda")

generator = torch.Generator(device="cuda").manual_seed(42)
image = pipe(
    class_labels="golden retriever",
    height=256,
    width=256,
    num_inference_steps=250,
    guidance_scale=1.5,
    generator=generator,
).images[0]
image.save("demo.png")

Load a variant subfolder (e.g. ./FiTv2-XL-2-256), not the repo root. For FiTv1, use ./FiTv1-XL-2-256 with the same call pattern (FiTPipeline + DDPM scheduler).

Repo layout

text

BiliSakura/FiT-diffusers/
├── README.md
├── FiTv1-XL-2-256/
├── FiTv2-XL-2-256/
├── FiTv2-3B-2-256/
├── FiTv2-XL-2-512/
└── FiTv2-3B-2-512/
    ├── README.md
    ├── model_index.json
    ├── pipeline.py
    ├── demo.png
    ├── transformer/
    │   ├── config.json
    │   ├── fit_transformer_2d.py
    │   └── diffusion_pytorch_model.safetensors
    ├── vae/
    │   ├── config.json
    │   └── diffusion_pytorch_model.safetensors
    └── scheduler/
        └── scheduler_config.json