Model Card for QwenPaw-Flash-4B (Core ML, Apple Neural Engine)
A compiled Core ML build of QwenPaw-Flash-4B, packaged to run entirely on device
on the Apple Neural Engine (ANE). It targets macOS (Apple silicon), needs no
network at inference time, and streams tokens locally with the weights quantized
to 6 bits.
This repository holds the compiled model set only, not training or original
weights. It is a self-contained artifact that a companion on-device runtime
loads to drive generation. Nothing here calls out to a server.
Model Details
Model Description
QwenPaw-Flash-4B shares the Qwen3.5 hybrid design: of its 32 layers, 24 are
Gated DeltaNet linear-attention (state-space) blocks and 8 are full
softmax-attention blocks, interleaved "three linear, one attention". The linear
blocks carry a small fixed-size recurrent state instead of a growing key/value
cache, so memory stays flat as context grows, while the periodic attention
blocks preserve exact long-range recall. It is the larger sibling of the 0.8B
and 2B builds, with the same vocabulary and a deeper, wider network. This build
converts the upstream model to Core ML and palettizes the weights to 6 bits for
on-device use.
- Developed by: AgentScope / QwenPaw authors (base model); Core ML
conversion by leok7v
- Model type: Hybrid Gated DeltaNet + attention causal language model,
converted to Core ML (mlprogram)
- Language(s): English and the languages of the base model
- License: Apache 2.0 (inherited from the base model)
- Converted from model:
agentscope-ai/QwenPaw-Flash-4B
Model Sources
Uses
Direct Use
On-device assistants and chat, drafting, and summarization, in offline or
privacy-sensitive settings where prompts must not leave the device. At 4B
parameters it is more capable than the 0.8B and 2B builds but heavier: it is
intended for Apple silicon Macs with adequate memory, not phones. The set is
driven by an on-device runtime that runs prompt ingestion (prefill) as a batched
pass on the Neural Engine and generates tokens one at a time, carrying the
linear-attention state forward and reading the attention layers from a paged
key/value cache.
Out-of-Scope Use
Not for tasks requiring verified factual accuracy, and not a basis for legal,
medical, financial, or safety decisions. Not intended for high-stakes automated
decisions without human review.
Bias, Risks, and Limitations
A language model predicts plausible text; it does not verify facts and has no
understanding of truth. Output can be confident and still be wrong, incomplete,
or biased, and 6-bit quantization can widen small errors relative to the base
model. The base model's biases and limitations carry over; see the base model
card.
Recommendations
Treat every response critically and check important output against a trusted
source. This model defaults to emitting a "thinking" prefix; preserve that
per-model default when integrating a chat template.
How to Get Started with the Model
The repository contains a set of compiled .mlmodelc programs plus the
tokenizer. It is designed to be loaded by an on-device runtime that implements
paged key/value attention on the host between the Neural Engine trunk programs;
it is not a single .mlpackage you call with one prediction. The included
chat_template.jinja documents the chat format (roles, the "thinking" prefix,
tool-call rendering).
Technical Specifications
Model Architecture and Objective
| Property | Value |
|---|
| Objective | Autoregressive next-token prediction |
| Hidden size | 2560 |
| Layers | 32 (24 Gated DeltaNet + 8 attention) |
| Attention heads | 16 query / 4 key-value (grouped-query) |
| Vocabulary | 248320 |
| Context length | up to 262144 tokens |
| Weight format | 6-bit palettized (per-grouped-channel k-means) |
| Activations | fp16 |
Files
mf0of9.mlmodelc ... mf8of9.mlmodelc - the transformer trunk, cut into 9
programs around the attention layers; each is a multifunction bundle whose
"prefill" (batched) and "decode" (single-token) graphs share one quantized
weight blob, so the weights are resident once.head.mlmodelc - the tied vocabulary matrix, shared by the output projection
and the token-embedding lookups (one quantized copy, three functions).tile.mlmodelc, tile_prefill.mlmodelc, tile_carry.mlmodelc - weightless
flash-attention kernels for the paged key/value cache.tokenizer.json, tokenizer_config.json, chat_template.jinja - the
byte-level BPE tokenizer and chat template.
Compute Infrastructure
- Hardware: Apple silicon Mac with a Neural Engine.
- Software: macOS 15+. The first launch compiles the programs for the Neural
Engine once; the OS caches that build, so later launches are fast.
Citation
Please cite the upstream QwenPaw-Flash-4B model. See the base model card at
agentscope-ai/QwenPaw-Flash-4B
for the canonical reference and license.
Model Card Contact
leok7v on Hugging Face.
