You can deploy this model and start earning money today!

Werve/
JOSIE-1.1-4B-Instruct-litert-lm

Name: JOSIE-1.1-4B-Instruct-litert-lm
Availability: InStock

litert-lm

edge-gallery

on-device

Goekdeniz-Guelmez/JOSIE-1.1-4B-Instruct

finetune

apache-2.0

Views

No views yet

JOSIE-1.1-4B-Instruct_DYNAMIC WI4 AFP32 — LiteRT-LM

On-device .litertlm conversion of Goekdeniz-Guelmez/JOSIE-1.1-4B-Instruct for the LiteRT-LM runtime. Runs completely offline on Android, Pixel, and Google Tensor / Mali-based devices.

Spec	Value
Base	`Goekdeniz-Guelmez/JOSIE-1.1-4B-Instruct`
Format	`.litertlm` (LiteRT-LM bundle)
Quantization	DYNAMIC WI4 AFP32
Context	4096 tokens
Size	2.09 GB
Modalities	Text only (vision/audio weights stripped)
License	Apache 2.0

What This Model Is

This is a community-packaged variant of the model provided for on-device inference via the LiteRT-LM engine.

Key characteristics:

Text-only — image and audio understanding weights were removed to keep the bundle small and optimized for text tasks.
4096 context window — configured for local inference memory efficiency.
WI4 quantized — optimized for mobile storage with balanced precision.
On-device — runs completely offline on supported Android / Pixel / Tensor devices.

How to Use

Google AI Edge Gallery (Pixel / Android)

Install Google AI Edge Gallery or a compatible fork.
Add model via HuggingFace URL: Werve/JOSIE-1.1-4B-Instruct-litert-lm
Download the 2.09 GB bundle over WiFi.
Chat offline — no data leaves your device.

LiteRT-LM CLI (macOS / Linux / Desktop)

bash

pip install litert-lm
litert-lm run --from-huggingface-repo Werve/JOSIE-1.1-4B-Instruct-litert-lm \
              JOSIE-1.1-4B-Instruct_DYNAMIC WI4 AFP32.litertlm \
              --prompt "Your prompt here" \
              --backend gpu

Technical Details

Export Parameters

Quantization: dynamic_wi4_afp32
Context Length: 4096
Export Library: litert-torch 0.9.1+

Weight Extraction

Only text-decoder safetensors were retained. Multimodal components (vision/audio) were stripped to reduce total size from the original checkpoint.

Credits & Attribution

Original Model: Goekdeniz-Guelmez/JOSIE-1.1-4B-Instruct
LiteRT-LM packaging: Werve

This is an unofficial community build. It is not published by Google or the official LiteRT team.