Views
No views yet
vision_encoder.pte - Vision encoder (SigLIP-B/16)modality_projector.pte - Projects vision features to language spacelanguage_decoder_prefill.pte - Language decoder prefill phaselanguage_decoder_decode.pte - Language decoder decode phase with KV cachetoken_embedding.pte - Token embedding lookuplm_head.pte - Language model output headconfig.json - Model configuration1# Install dependencies
2pip install executorch torch pillow transformers
3
4# Download model
5huggingface-cli download infil00p/nanoVLM-230M-8k-executorch --local-dir executorch_models
6
7# Clone nanoVLM repo for test script
8git clone https://github.com/huggingface/nanoVLM
9cd nanoVLM
10
11# Run inference test
12python test_executorch_pte.py --model_dir ../executorch_models --image assets/image.png1from executorch.extension.pybindings.portable_lib import _load_for_executorch
2import torch
3
4# Load models
5vision_encoder = _load_for_executorch("vision_encoder.pte")
6modality_projector = _load_for_executorch("modality_projector.pte")
7prefill_decoder = _load_for_executorch("language_decoder_prefill.pte")
8decode_decoder = _load_for_executorch("language_decoder_decode.pte")
9token_embedding = _load_for_executorch("token_embedding.pte")
10lm_head = _load_for_executorch("lm_head.pte")
11
12# Run inference (see test_executorch_pte.py for full example)
13# 1. Encode image with vision_encoder
14# 2. Project with modality_projector
15# 3. Combine with text embeddings from token_embedding
16# 4. Run prefill_decoder for initial KV cache
17# 5. Autoregressive decode with decode_decoder
18# 6. Get logits with lm_head"A close-up photograph captures a tabby cat with a focused gaze, sitting on a patterned surface. The cat's fur exhibits a mix of dark..."
python export_executorch.py --checkpoint lusxvr/nanoVLM --output_dir executorch_models --quantizetorchao