You can deploy this model and start earning money today!

barha/
granite-switch-4.0-350m-demo-onnx

Name: granite-switch-4.0-350m-demo-onnx
Availability: InStock

Views

No views yet

granite-switch-4.0-350m-demo (ONNX / transformers.js)

ONNX export of a Granite Switch model — base Granite with per-token LoRA adapter switching driven by control tokens — packaged to load on transformers.js.

adapters: 3 (cti-technique-mapping, text-to-json, genai-attack-vector)
layers: 28, kv-heads: 4, head-dim: 64
vocab: 100355

Available dtypes

fp32 — full precision (onnx/model.onnx)
int8 — onnx/model_int8.onnx

Loading (native transformers.js architecture)

This repo declares its true model_type: granite_switch. The Granite Switch web runtime ships a self-registering shim that teaches transformers.js the GraniteSwitchForCausalLM architecture, so it loads via the standard AutoModelForCausalLM.from_pretrained. Granite Switch selects adapters via a causal, cumulative switch attention; the shim's custom forward threads that state (past_switch_key0 / past_switch_val0) across decode steps — the generic loop alone is not sufficient.

// import the shim once (registers granite_switch), then use the re-exported API
import { AutoModelForCausalLM, loadGraniteSwitch } from "granite-switch-web/src/granite-switch-register.js";

// loadGraniteSwitch wires the external-data sidecar (model.onnx.data) for you:
const gs = await loadGraniteSwitch("barha/granite-switch-4.0-350m-demo-onnx", { dtype: "fp32" });
const out = await gs.generate({ inputs, max_new_tokens: 16, do_sample: false });

gs_onnx.json carries the runtime metadata (layer/head counts, adapter token ids) the decode loop needs.

Provenance

Exported with granite_switch.onnx.export + granite_switch.onnx.package. See the project docs (docs/ONNX_BROWSER_PORT.md) for the export pipeline and parity gates (HF-vs-ONNX max|diff| ~1e-4, argmax 100% on the source model).