Views
No views yet
A Hugging Face-compatible wrapper around the FastViT-HD vision backbone from
FastVLM: Efficient Vision Encoding for Vision-Language Models (Apple CVPR 2025).
This repo exposes only the image encoder – no text tower, no projection head – so you can plug it into any downstream pipeline that needs per-image embeddings.
transformers.| Variant | #Params (enc.) | Output dim | Patch size | Global pool |
|---|---|---|---|---|
| FastViT-HD (this) | ~272 M | 3 072 | 64 | Yes |
1conda create --name fast-vit-hd python=3.10
2conda activate fast-vit-hd
3pip install torch torchvision transformers timm pillow1from transformers import AutoModel, AutoImageProcessor
2import torch, PIL.Image
3
4device = "cuda" # or "cpu" / "mps"
5
6model = AutoModel.from_pretrained(
7 "kevin510/fast-vit-hd", trust_remote_code=True
8).to(device).eval()
9
10processor = AutoImageProcessor.from_pretrained(
11 "kevin510/fast-vit-hd", trust_remote_code=True
12)
13
14img = PIL.Image.open("your_image.jpg")
15px = processor(img, do_center_crop=False, return_tensors="pt")["pixel_values"].to(device) # (1,3,1024,1024)
16
17emb = model(px)
18print(emb.shape) # (1, D, 3072)FastViTImageEncoder extends PreTrainedModel; we keep the original GlobalPool2D head but replace the classifier by a 3 072 × 3 072 identity-mapped projection.llava-fastvithd_0.5b_stage3/fast_vit/fast_vit.pth.transformers ≥ 4.48 schema.1@inproceedings{fastvlm2025,
2 title = {FastVLM: Efficient Vision Encoding for Vision Language Models},
3 author = {Vasu, Pavan Kumar Anasosalu and Faghri, Fartash and Li, Chun-Liang et al.},
4 booktitle = {CVPR},
5 year = {2025}
6}mci.py code implementation is licensed according to Apple's LICENSE; it is a modified version of the original mci.py from the FastVLM repo. The underlying weights inherit the license provided by Apple in their LICENSE_MODEL; review that file before use. All other code in this repo is licensed according to Apache 2.0.Transformers.