TinySwallow-1.5B-Instruct-ONNX – AI Model by moeadham | AlphaNeural AI

You can deploy this model and start earning money today!

moeadham/
TinySwallow-1.5B-Instruct-ONNX

like

0

transformers.js

text-generation

SakanaAI/TinySwallow-1.5B-Instruct

Views

No views yet

TinySwallow-1.5B-Instruct (ONNX, q4 + int8 embed)

ONNX-quantized version of SakanaAI/TinySwallow-1.5B-Instruct for in-browser inference with transformers.js v3 + WebGPU.

File

File	Size	Notes
`onnx/model_q4.onnx`	~1.06 GB	4-bit MatMul (block_size=128, sym) + int8 per-row embeddings; everything inline.

Usage

js

import { pipeline } from "@huggingface/transformers";

const generator = await pipeline(
  "text-generation",
  "moeadham/TinySwallow-1.5B-Instruct-ONNX",
  { device: "webgpu", dtype: "q4" }
);
const out = await generator(
  [{ role: "user", content: "知識蒸留について簡単に教えてください。" }],
  { max_new_tokens: 256 }
);
console.log(out[0].generated_text);