This repository contains an experimental MLC/WebLLM conversion of prism-ml/Ternary-Bonsai-1.7B-unpacked. It is a browser-runtime artifact, not a new model, fine-tune, GGUF, MLX, or ONNX mirror.

The source checkpoint is Prism ML's unpacked FP16 Ternary Bonsai model. This conversion uses a local MLC bonsai_tq_f32 profile: symmetric 2-bit group quantization with uint32 storage, group size 128, and FP32 scales. The encoded values represent the ternary lane -scale, 0, and +scale.

Artifact Summary

Field	Value
Source checkpoint	`prism-ml/Ternary-Bonsai-1.7B-unpacked`
Architecture	Qwen3-shaped decoder
MLC model type	`qwen3`
Quantization	`bonsai_tq_f32`
Quantized storage	2-bit symmetric group quantization in `uint32`
Conversation template	`qwen3_nothink`
Context window in config	`32768`
Prefill chunk in config	`2048`
Total parameters	1,720,028,160
Quantized parameter size	0.451 GB
Bits per parameter	2.252
Parameter shards	13
Artifact size	about 460 MB
WebGPU library	`libs/ternary-bonsai-1.7b-bonsai_tq_f32-webgpu.wasm`

Runtime Requirement

This artifact requires an MLC/WebLLM runtime with the local bonsai_tq_f32 quantization profile registered. It is not expected to load in an unmodified upstream WebLLM build until this profile is upstreamed or otherwise carried in the runtime.

This first ternary path uses MLC's group-quantized graph path. It is a compact WebGPU artifact and a correctness/release milestone, but it is not yet a custom fused ternary matmul kernel. Benchmark it before making speed claims.

WebLLM Configuration

const appConfig = {
  model_list: [
    {
      model: "https://huggingface.co/welcoma/Ternary-Bonsai-1.7B-bonsai_tq_f32-MLC/resolve/main/",
      model_id: "Ternary-Bonsai-1.7B-tq-MLC",
      model_lib:
        "https://huggingface.co/welcoma/Ternary-Bonsai-1.7B-bonsai_tq_f32-MLC/resolve/main/libs/ternary-bonsai-1.7b-bonsai_tq_f32-webgpu.wasm",
      overrides: {
        context_window_size: 4096,
        prefill_chunk_size: 512,
      },
    },
  ],
};

The smaller override values above are intended for local browser smoke tests. Increase them only after measuring browser memory and cache behavior on the target device.

Validation

The artifact was converted and WebGPU-compiled on the GCP MLC/WebLLM builder VM, not on a local laptop.

Source: prism-ml/Ternary-Bonsai-1.7B-unpacked
Quantization: bonsai_tq_f32
Quantization profile: int2 values, uint32 packed storage, FP32 scales
Conversion peak RAM: 3.204 GB on CPU
WebGPU compile completed successfully
Compile estimate without KV cache: 1886.70 MB
Compile estimate with 4K KV cache: 2782.70 MB

Limitations

This is an experimental runtime artifact, not a general transformers model checkpoint.
This repo does not claim the same runtime performance as Prism ML's native MLX 2-bit release.
Quality evaluation is limited to conversion and WebGPU compile checks; no benchmark score is claimed by this repository.
Browser success depends on WebGPU support, available GPU memory, cache quota, and a compatible patched WebLLM runtime.

Provenance

Original model by Prism ML:

MLC/WebLLM conversion by welcoma.

welcoma/Ternary-Bonsai-1.7B-bonsai_tq_f32-MLC

Ternary-Bonsai-1.7B bonsai_tq_f32 for MLC/WebLLM