You can deploy this model and start earning money today!

NghiaNguyen1529/
octen-embedding-0.6b-directml-onnx

Name: octen-embedding-0.6b-directml-onnx
Availability: InStock

sentence-transformers

Views

No views yet

Octen-Embedding-0.6B — DirectML ONNX

Patched ONNX export of Octen/Octen-Embedding-0.6B that runs on DirectML (Windows GPU via ONNX Runtime).

What was fixed

The original torch.onnx.export (dynamo) produces val_41 = [-1] used in Reshape shapes for multi-head attention (GQA: 16 Q heads, 8 KV heads). DirectML's execution provider cannot resolve symbolic -1 at graph-capture time.

Fix: Replace [-1] with four concrete head-count constants (16 for Q, 8 for K, 8 for V, 2048 for attention output) and reconnect 84 Reshape consumer nodes.

See fix_octen_dml.py for the full patch script.

Files

model.fp16.onnx — ONNX graph proto (4 MB)
model.fp16.onnx.data — external weights (1.1 GB, fp16)
tokenizer.json — Qwen2 tokenizer
config.json — model config (max_position_embeddings=32768)
fix_octen_dml.py — reproduction script

Usage

python

import onnxruntime as ort

session = ort.InferenceSession(
    "model.fp16.onnx",
    providers=["DmlExecutionProvider"],
)

Quality

Dataset	R@5	R@10	MRR
esp32 (smoke)	0.930	0.950	0.810
autosar	0.678	0.774	0.552

Identical to CPU fp16 reference — patch preserves quality exactly.

NghiaNguyen1529/octen-embedding-0.6b-directml-onnx