File	Precision	Size	Input
`jina-clip-v2-text-fp16.onnx`	FP16	1.12 GB	`input_ids`: int64 `[batch, sequence]`
`jina-clip-v2-text-fp32.onnx` (+`.data`)	FP32	2.24 GB	`input_ids`: int64 `[batch, sequence]`
`jina-clip-v2-text-int8.onnx`	Dynamic INT8	564 MB	`input_ids`: int64 `[batch, sequence]`
`jina-clip-v2-vision-fp16.onnx`	FP16	610 MB	`pixel_values`: float32 `[batch, 3, 512, 512]`
`jina-clip-v2-vision-fp32.onnx` (+`.data`)	FP32	1.22 GB	`pixel_values`: float32 `[batch, 3, 512, 512]`
`jina-clip-v2-vision-int8.onnx`	Dynamic INT8	313 MB	`pixel_values`: float32 `[batch, 3, 512, 512]`

Every model is self-contained (FP32 uses an external-data companion file), supports dynamic batch sizes, and returns normalized float32 embeddings named embeddings with shape [batch, 1024]. The text models also support dynamic sequence lengths.

Which runtime should I use?

Measured on RTX 2070 (sm_75) unless noted. Full methodology: GitHub results.md.

scenario	what to use	measured
GPU, Turing (sm_75) or newer, fastest	TensorRT engine built from the FP16 text graph via this repo's `scripts/build_trt_engine.py`	b1×1024 25 ms, b32×1024 0.80 s, flat ~3 GiB VRAM
GPU via ONNX Runtime	FP16 models (text: apply the Einsum→MatMul rewrite first — −25%)	b1×1024 76–97 ms, b32×1024 2.4–2.6 s
GPU, portable PyTorch, long context	fp16 torch + `sdpa_patch.py`	b32×1024 1.70 s, flat ~2.65 GB
CPU	dynamic INT8 models	text b8x128 tokens 0.74 s vs 3.14 s FP16 (i9-7980XE, ORT 1.29)

Notes:

TensorRT fuses each attention layer into a single flash-attention-style kernel (_gemm_mha_v2) with linear memory; engines are hardware-specific and bound to the built optimization profile (default [1,8]..[32,2048]; pad shorter sequences with pad-token id 1, which the internal mask makes semantically neutral). GTX 1080 Ti / Pascal cannot run TensorRT 10+ — use ONNX Runtime there.
FP16 ONNX is not recommended for CPU inference: CPU providers promote unsupported FP16 ops or insert casts, increasing latency and memory.

Preprocessing and inference

Use the tokenizer and image processor from the base model:

python

import onnxruntime as ort
from transformers import AutoImageProcessor, AutoTokenizer

base_model = "jinaai/jina-clip-v2"
tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
image_processor = AutoImageProcessor.from_pretrained(base_model, trust_remote_code=True)

text_session = ort.InferenceSession("jina-clip-v2-text-int8.onnx")
tokens = tokenizer(["A photo of a cat"], padding=True, truncation=True, return_tensors="np")
text_embeddings = text_session.run(["embeddings"], {"input_ids": tokens.input_ids})[0]

# `images` is a list of PIL images.
vision_session = ort.InferenceSession("jina-clip-v2-vision-int8.onnx")
pixels = image_processor(images=images, return_tensors="np").pixel_values
image_embeddings = vision_session.run(["embeddings"], {"pixel_values": pixels})[0]

Matryoshka dimensions

The trained dimensions are 32, 64, 128, 256, 512, 768, and 1024. To reproduce the base model's truncate_dim=N behavior, truncate the prefix and normalize it again:

python

import numpy as np

embedding = embedding[:, :dimension]
embedding /= np.linalg.norm(embedding, axis=1, keepdims=True)

Validation

Both INT8 graphs were re-quantized on 2026-09-01 with per-channel weight scales and ORT's reduce_range; the artifacts published before that date were per-tensor and are not faithful — a single caption embedded at mean cosine 0.70 to the base model. If you cached the earlier files, re-download them.

The text encoder is scored on 424 real captions against ground-truth PyTorch fp32, on two metrics, because neither catches the other's failure mode: per-vector cosine is absolute and per-document, retrieval ranking is relative and cross-document, and this export has shipped a build that passed each while failing the other.

Dimension	Mean cosine	Worst caption	Ranking
64	0.9929	0.9699	OK
128	0.9899	0.9694	OK
256	0.9866	0.9639	OK
512	0.9846	0.9597	OK
768	0.9839	0.9589	OK
1024	0.9838	0.9586	OK

The FP16 and FP32 graphs score 1.0000 on the same check. Ranking is a 20-profile retrieval fixture in which the mean-pooled vectors of 4 relevant profiles must outrank 16 irrelevant ones; dim=32 ranks wrongly for every build including FP16 and is a property of that fixture, not of any encoder. Reproduce with jina-clip-verify --text-model <graph> from the GitHub repository.

Vision INT8 scores mean cosine 0.9874 (worst 0.9831) at 512 dimensions and 0.9855 (worst 0.9809) at 1024 against the FP16 vision graph on synthetic structured images; a real-image vision gate is still open.

INT8 CPU performance

ONNX Runtime 1.29, Intel i9-7980XE, ORT_ENABLE_BASIC, mean of 5 runs:

Shape	INT8	FP16
b1x128	0.13 s	0.53 s
b8x128	0.74 s	3.14 s
b8x512	3.53 s	12.48 s

Peak resident memory: 2.6 GiB INT8 vs 5.0 GiB FP16. The text input remains dynamic up to 8192 tokens, although ordinary full attention still has quadratic memory use; the TensorRT path removes that quadratic term on supported GPUs.

License and attribution

This is a quantized and structurally split derivative of jinaai/jina-clip-v2. It retains the base model's CC BY-NC 4.0 license and non-commercial restriction. Refer to the base model card for intended use, limitations, training details, and complete attribution.

satorean/jina-clip-v2-split-onnx

Jina CLIP v2 split ONNX