You can deploy this model and start earning money today!

skkut/
qwen3-embedding-8b-q4f16_1-MLC

Name: qwen3-embedding-8b-q4f16_1-MLC
Availability: InStock

Views

No views yet

Qwen3-Embedding-8B-q4f16_1-MLC

MLC-compiled Qwen3-Embedding-8B (Apache-2.0) in web-llm 0.2.84's format for WebGPU in-browser embedding. Built locally with mlc-llm (tvmjs 0.24.0-dev3 / mlc-ai 0.26.dev246 generation) + emsdk 3.1.56.

Quantization: q4f16_1 · context: 2048 · prefill chunk: 2048 · max batch: 2
Embedding dim: 4096 (last-token pooling, L2-normalized — done in the compiled model)
WebGPU: works on NVIDIA (Dawn); grid/buffer limits sized for batch 2/2048
VRAM: ~6.9 GB (compile estimate 6620 MB + KV) — best on 16 GB-class cards
Why q4f16_1? the fp8 8-bit path cannot build for WebGPU (TVM's wasm codegen only emits f16/f32), so q4f16_1 is the effective VRAM-friendly build here: 16.5 GB fp16 → ~4 GB.

Build note: the 8B checkpoint declares tie_word_embeddings=false but ships no lm_head.weight — the build ties the head to the embedding table (unused for embeddings anyway).

Files

File	Purpose
`mlc-chat-config.json`	model config (ctx 2048, chunk 2048, max_batch 2)
`tensor-cache.json`	weight manifest
`params_shard_0.bin … params_shard_109.bin`	quantized weights (~4.0 GB)
`tokenizer.json`, `tokenizer_config.json`, `vocab.json`, `merges.txt`	Qwen3 tokenizer
`qwen3-embedding-8b-q4f16_1-ctx2048_batch2-webgpu.wasm`	the model library (wasm)

Use in web-llm (0.2.84)

const record = {
  model: 'https://huggingface.co/skkut/qwen3-embedding-8b-q4f16_1-MLC/resolve/main/',
  model_id: 'Qwen3-Embedding-8B-q4f16_1-MLC',
  model_lib: 'https://huggingface.co/skkut/qwen3-embedding-8b-q4f16_1-MLC/resolve/main/qwen3-embedding-8b-q4f16_1-ctx2048_batch2-webgpu.wasm',
  model_type: 1, // EmbeddingModelType
  vram_required_MB: 6900,
};

Query-side texts should use the Qwen3 instruction prefix (Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: …).