You can deploy this model and start earning money today!

kucukkanat/
LFM2.5-Encoder-350M-Policy-Linter-ONNX

Name: LFM2.5-Encoder-350M-Policy-Linter-ONNX
Availability: InStock

Views

No views yet

LFM2.5-Encoder-350M-Policy-Linter-ONNX

ONNX export of LiquidAI/LFM2.5-Encoder-350M-Policy-Linter, quantized to run fully in the browser through transformers.js. No inference server: the weights are fetched once, cached, and every forward pass happens in the tab.

A zero-shot policy linter. Rules are ordinary prose supplied at call time. One pass scores every token against every rule, so adding a rule costs a few tokens rather than another model.

All credit for the model itself goes to Liquid AI. This repository contains only a re-export; the weights are unchanged apart from quantization, and the original LFM Open License v1.0 applies.

Try it in your browser → — no install, no API key.

Tooling, demo and the export pipeline: https://github.com/kucukkanat/lfm-encoders

Files

dtype	File	Size
`q8`	`onnx/model_quantized.onnx`	357 MB
`q4`	`onnx/model_q4.onnx`	449 MB

The graph takes input_ids + attention_mask, is dynamic in batch and sequence, and returns token_proj and rule_proj.

Usage

import { AutoTokenizer, PreTrainedModel, Tensor } from "@huggingface/transformers";

const id = "kucukkanat/LFM2.5-Encoder-350M-Policy-Linter-ONNX";
const tokenizer = await AutoTokenizer.from_pretrained(id);
const model = await PreTrainedModel.from_pretrained(id, { dtype: "q8" });

const { input_ids } = tokenizer("some text");
const out = await model({
  input_ids,
  attention_mask: new Tensor("int64", new BigInt64Array(input_ids.dims[1]).fill(1n), input_ids.dims),
});

PreTrainedModel rather than AutoModel is deliberate: this is a plain "feed the named inputs, read the named outputs" session, not one of transformers.js's built-in architectures.

Prompt format

Both projection towers expect one string laid out exactly like this — the model was trained on it and the character arithmetic that locates each label depends on it byte for byte:

Policy:
- label one
- label two

Text:
<the text>

token_proj is the query tower and rule_proj the key tower, both 256-d and emitted per token. Pool the tokens covering each label to get its vector. Pooling after projecting is exact rather than an approximation: both towers are affine, and an affine map commutes with a mean — which is what keeps one static graph usable for any number of labels.

Scoring is a dot product scaled by 1/sqrt(256) plus a bias, through a sigmoid per (token, rule) pair.

@lfm-encoder/tasks implements all of this, including the character-offset reconstruction transformers.js does not provide.

Accuracy

Measured from JavaScript against the fp32 PyTorch reference. Δ is the largest absolute difference in a final probability.

dtype	max Δ	mean Δ	threshold flips (6 cases)
`fp32`	8.4e-4	3.4e-4	0
`q8`	0.5241	0.2328	3
`q4`	0.3698	0.1818	3

Notes

q8 is smaller on disk but uses more browser RAM than fp32 and runs slower: onnxruntime's WASM kernels compute in float, so quantized weights are unpacked at session load. Quantization here buys download size, not speed or memory.
Budget roughly 1.5 GB of RAM per resident model, and expect a tab to hold its high-water mark until reloaded.
fp16 / q4f16 are deliberately absent: RMSNorm's variance overflows fp16 on this architecture and every hidden state collapses to zeros.