You can deploy this model and start earning money today!

sona-forge/
clip-vit-h-14-image-fp16

Name: clip-vit-h-14-image-fp16
Availability: InStock

Views

No views yet

Sona Forge — CLIP ViT-H/14 image encoder (ONNX FP16)

Vision-tower-only export of OpenCLIP ViT-H/14 used by IP-Adapter to produce image embeds for SD 1.5 conditioning. Used by the Sona Forge Android app. Pair with sona-forge/sd15-ipadapter-fp16.

ONNX shape

Input	Shape	dtype	Notes
`pixel_values`	`[batch, 3, 224, 224]`	FP16	center-cropped + NEAREST-resized + CLIP-mean/std-normalized

Output	Shape	dtype	Notes
`image_embeds`	`[batch, 1024]`	FP16	primary output
`onnx::Gather_4496`	`[batch, seq, hidden]`	FP16	secondary `last_hidden_state` passthrough; downstream consumers query by name (`image_embeds`) and ignore this

How it was made

Pinned conversion environment:

Package	Version
transformers	4.40.0
torch	2.3.0
onnx	1.16.0
numpy	<2 (ABI compat)

Conversion sequence:

Load transformers.CLIPVisionModelWithProjection.from_pretrained("h94/IP-Adapter", subfolder="models/image_encoder", torch_dtype=torch.float16). Note: this loads the OpenCLIP ViT-H/14 weights bundled inside the IP-Adapter repo as a convenience copy.
model.eval().
torch.onnx.export(model, dummy_pixel_values, opset_version=17, input_names=["pixel_values"], output_names=["image_embeds"], dynamic_axes={"pixel_values": {0: "batch"}, "image_embeds": {0: "batch"}}).

The text branch is not exported. Phase 6 spike characterisation used a deterministic synthetic 512×512 fixture; CLIP image-embeds norm = 21.9 (in the typical 15–25 range for natural portraits).

Files

File	Size	sha256
`model.onnx`	1,264,856,075 B (1206 MB)	`038078ba22b67e57ec2ca5b466e0699f30e14a9613054be8c7fb24e537987689`

Licence

OpenCLIP ViT-H/14 — MIT. Original training data is LAION-2B.

Preprocessing

The Sona Forge Android-side ClipImageEncoder uses NEAREST interpolation in the resize step, not BILINEAR. Reasoning: Pillow ≥ 10.0's Image.BILINEAR is anti-aliased by default and produces a small but consistent cosine drift (~0.04) versus a Kotlin pure-bilinear implementation. NEAREST on both sides clears the 0.999 cosine threshold the in-tree JVM golden test asserts. If you re-use this ONNX with a different preprocessor, verify cosine match against your own reference embedding.

Memory footprint

~1.2 GB FP16 on disk; ORT CPU EP promotes to FP32 at session load (~2.4 GB resident). On Android (NNAPI / XNNPack), FP16 runs natively.

Usage

python

import onnxruntime as ort
import numpy as np
from PIL import Image

clip = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])

CLIP_MEAN = np.array([0.48145466, 0.4578275, 0.40821073], dtype=np.float32)
CLIP_STD = np.array([0.26862954, 0.26130258, 0.27577711], dtype=np.float32)

img = Image.open("portrait.png").convert("RGB")
w, h = img.size
side = min(w, h)
img = img.crop(((w - side) // 2, (h - side) // 2, (w + side) // 2, (h + side) // 2))
img = img.resize((224, 224), Image.NEAREST)
arr = (np.asarray(img, dtype=np.float32) / 255.0 - CLIP_MEAN) / CLIP_STD
arr = arr.transpose(2, 0, 1)[None, ...].astype(np.float16)  # NCHW FP16

image_embeds = clip.run(["image_embeds"], {"pixel_values": arr})[0]  # (1, 1024)

Provenance

Original weights: laion/CLIP-ViT-H-14-laion2B-s32B-b79K.
Bundled-copy used by this conversion: h94/IP-Adapter/models/image_encoder.

sona-forge/clip-vit-h-14-image-fp16