LiquidAI/LFM2.5-Encoder-230M converted to LiteRT (.tflite) for on-device inference. A multilingual (15 languages) bidirectional encoder on the LFM2 hybrid backbone — gated short-convolutions plus grouped-query attention — for embeddings, retrieval, classification heads, and masked-token prediction, fully offline on CPU. This is the lightweight sibling of the 350M model, for tighter latency and memory budgets.

Model description

File	Recipe	Size	Target
`LFM2.5-Encoder-230M_wi8fc.tflite`	int8 dynamic-range (linears + embedding, convs float)	246 MB	mobile + desktop
`LFM2.5-Encoder-230M_fp16.tflite`	fp16 weights, float compute	463 MB	desktop — XNNPACK's per-signature fp32 unpacking exceeds iPhone memory limits

All signatures take batch-1, right-padded static shapes: input_ids int32 [1, S] and attention_mask int32 [1, S] (1 = real token, 0 = pad).

Signature	Output
`encode_64` / `encode_128` / `encode_256` / `encode_512`	`last_hidden_state` float32 `[1, S, 1024]`, zeroed at padded positions
`mlm_128`	masked-LM logits float32 `[1, 128, 65536]`

Padded positions are fully masked inside the graph, in both the convolution path and attention, so the output at valid positions does not depend on how much padding follows: encode_64, encode_128 and encode_256 agree bitwise on the same sentence and match the unpadded PyTorch reference.

For the larger sibling see LFM2.5-Encoder-350M.

How to use

1. Install dependencies

pip install ai-edge-litert numpy tokenizers huggingface_hub

2. Save the script below as embed.py:

python

#!/usr/bin/env python3
"""Embed sentences with litert-community/LFM2.5-Encoder-230M and rank them by similarity."""
import argparse

import numpy as np
from ai_edge_litert.interpreter import Interpreter
from huggingface_hub import hf_hub_download
from tokenizers import Tokenizer

REPO = "litert-community/LFM2.5-Encoder-230M"
MODEL_FILE = "LFM2.5-Encoder-230M_wi8fc.tflite"


def embed(runner, tokenizer, text, seq_len):
    """Mean-pools the encoder states over the real tokens into one vector."""
    ids = tokenizer.encode(text).ids
    if len(ids) > seq_len:
        raise SystemExit(f"{len(ids)} tokens exceed --seq-len {seq_len}")
    input_ids = np.zeros((1, seq_len), np.int32)
    attention_mask = np.zeros((1, seq_len), np.int32)
    input_ids[0, : len(ids)] = ids
    attention_mask[0, : len(ids)] = 1
    states = list(runner(input_ids=input_ids, attention_mask=attention_mask).values())[0]
    vector = states[0, : len(ids)].mean(axis=0)
    return vector / np.linalg.norm(vector)


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--query", required=True, help="The sentence to match.")
    parser.add_argument("--candidate", action="append", required=True,
                        help="A candidate sentence, repeatable.")
    parser.add_argument("--seq-len", type=int, default=128,
                        choices=[64, 128, 256, 512])
    parser.add_argument("--threads", type=int, default=8)
    args = parser.parse_args()

    model_path = hf_hub_download(REPO, MODEL_FILE)
    tokenizer = Tokenizer.from_file(hf_hub_download(REPO, "tokenizer.json"))
    interpreter = Interpreter(model_path=model_path, num_threads=args.threads)
    runner = interpreter.get_signature_runner(f"encode_{args.seq_len}")

    query = embed(runner, tokenizer, args.query, args.seq_len)
    scored = [(float(query @ embed(runner, tokenizer, c, args.seq_len)), c)
              for c in args.candidate]
    for score, text in sorted(scored, reverse=True):
        print(f"{score:6.3f}  {text}")


if __name__ == "__main__":
    main()

3. Run it

bash

python embed.py --query "What is your refund policy?" \
  --candidate "Our refund policy allows 30 days" \
  --candidate "Steps to recover a forgotten login" \
  --candidate "The weather in Osaka is mild in spring"

 0.717  Our refund policy allows 30 days
 0.668  Steps to recover a forgotten login
 0.576  The weather in Osaka is mild in spring

Mean-pooling the raw encoder states is the simplest sentence representation and is what the numbers above use; for retrieval at quality you would normally train a pooling head or fine-tune on your own pairs. For masked-token prediction use the mlm_128 signature and read the logits at the [MASK] position.

On Android/iOS use the LiteRT runtime's SignatureRunner APIs with the same signature names; the tokenizer is the standard Hugging Face tokenizer.json, which the Rust/Swift/Kotlin tokenizers bindings all read.

Performance

int8 (wi8fc) file, CPU only.

Device	Threads	`encode_128`	`encode_512`	`mlm_128`
Apple M4 Max (macOS)	8	28.1 ms	87.4 ms	36.2 ms
iPhone 17 Pro	6	27 ms	93 ms	47 ms

Mac figures are the median of 20 warm runs (ai-edge-litert 2.1.6, XNNPACK, otherwise idle machine). iPhone figures come from the on-device gate (TFLite C API + SignatureRunner + XNNPACK); each is the last of three consecutive measurements, not a median.

Budget for one slow first call. The first inference after loading pays a one-time graph preparation. On the Mac that first call took 592 ms against a 28.1 ms steady state; on the iPhone the three consecutive encode_128 measurements were 43, 29 and 27 ms, and the three encode_512 measurements were 109, 93 and 93 ms. Model load was 0.98 s on the iPhone, with a peak footprint of about 1.0 GiB.

Those three consecutive values are not a language effect — the signatures are fixed-shape, so every input of a given signature costs the same. Measured warm on the Mac with the run order reversed, one signature takes 36.4 / 37.3 / 36.6 ms on English, Japanese and Arabic sentences of 17, 21 and 27 tokens.

Thread count matters more than anything else here: at the interpreter default this model measures 41.6 ms and 174.5 ms for encode_128 and encode_512, against 30.0 ms and 94.9 ms at 8 threads on the same run.

Accuracy note

Parity against the PyTorch fp32 reference over 16 sentences covering all 15 supported languages — mean-pooled sentence-embedding cosine against the original Lfm2BidirectionalModel, plus top-5 fill-mask agreement on English, French, German and Japanese cloze prompts:

Variant	Pooled cosine (min / mean)	Per-token correlation (min)	Fill-mask
fp16	1.000000 / 1.000000	0.999999	top-5 sets identical (4/4 prompts)
int8 (`wi8fc`)	0.994781 / 0.998148	0.986881	top-1 on 4/4, at least 3/5 top-5 overlap on all

On the iPhone 17 Pro the int8 file reproduces the Mac outputs bit-exactly — cosine 1.000000, max absolute difference 0.0 — across every tested language and signature.

Android (Pixel 8a)

Android figures use the standard TFLite benchmark_model on a Pixel 8a (Tensor G3, Android 16) — 5 warm-up runs then 20 timed runs, the signature selected explicitly with --signature_to_run_for, CPU at 4 threads.

Signature	GPU (OpenCL, previous export)	CPU (XNNPACK, 4 threads)
`encode_128`	232 ms	86 ms
`encode_512`	816 ms	329 ms
`mlm_128`	223 ms	86 ms

GPU works as of the 2026-08-13 re-export. The original export ran only 3 of 24 nodes on the OpenCL delegate; this re-export respells the one idiom mobile GPU delegates refuse — transformers' rank-5 repeat_kv expand — into an equivalent rank-4 matmul (outputs bitwise-identical on CPU), and the OpenCL delegate now takes 852/852 nodes across all 5 signatures. Measured with the LiteRT CompiledModel API (fp32 GPU precision, real token inputs, best of 3 warm runs): encode_512 10.4 ms on the Pixel 8a, cosine 0.9988 vs the fp32 desktop reference. On iPhone 17 Pro Metal the same file fully compiles and runs encode_512 in 104 ms (cosine 0.9988) — about CPU speed, so CPU (bit-exact) remains the iOS recommendation. Set the GPU precision to fp32 — at fp16 GPU precision this family's norm reductions overflow and every output is NaN. These CompiledModel timings are not comparable to the classic-delegate benchmark_model timings above (different GPU runtime).

Snapdragon NPU (Hexagon)

LFM2.5-Encoder-230M_fp16.tflite — neither accelerator produced a usable row on the S26. NPU: the benchmark process was killed, most likely out of memory. GPU: LiteRtException: Failed to compile model.
LFM2.5-Encoder-230M_wi8fc.tflite — the GPU runs it at 56.29 ms. The NPU does not — LiteRtException: Failed to compile model.

file	backend	compiled	inference (median / min)	load
`LFM2.5-Encoder-230M_wi8fc.tflite`	GPU (Adreno)	—	56.29 ms / 55.37 ms	12511 ms

Measured on a Samsung Galaxy S26 (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16) with LiteRT CompiledModel 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. The run held thermal status NONE throughout. Headroom 0.73–0.73, where 1.0 is the throttling threshold.

GPU wiring: GPU guide. NPU recipe: NPU guide.

License

LFM Open License v1.0 (see LICENSE, unchanged from the base model). Note the license's commercial-use threshold (Section 5). This repository redistributes converted Derivative Works of LiquidAI/LFM2.5-Encoder-230M with modification notices per Section 4; all credit for the model to Liquid AI.