Q4NX is FastFlowLM's native packed-quantization format - a rearranged Q4_1 layout tuned for the NPU matrix engine's tile sizes and memory access patterns. It is not a GGUF file and it does not run on llama.cpp or Ollama; it is meant exclusively for the FastFlowLM engine on AMD Ryzen AI NPUs.

Requirements

FastFlowLM >= 0.9.45 (flm CLI)
AMD Ryzen AI processor with XDNA2 (NPU2) - Strix Point / Ryzen AI 300 series or later
Linux with the XRT NPU stack installed
~15 GB of unified system memory (Q4NX weights + activations + KV cache)

Files

File	Purpose
model.q4nx	Quantized Q4NX weights
config.json	FastFlowLM model configuration
tokenizer.json	Tokenizer
tokenizer_config.json	Special tokens and chat template
chat_template.jinja	Chat template (optional)
vision_weight.q4nx	Vision tower weights (multimodal input)
audio_weight.q4nx	Audio tower weights
flm-add.py	Installer script - registers this model with FastFlowLM

Install and run

This repository works with flm-add, a small installer that copies the model into the FastFlowLM user directory and registers the tag. It never modifies the system FastFlowLM install.

pip install flm-add or uv tool install flm-add

bash

uv tool install flm-add
flm-add Atomic-Germ/Gemma4-E4B-GLM-NPU2 --tag gemma4-glm:4b --family gemma4
FLM_CONFIG_PATH="$HOME/.config/flm/model_list.json" FLM_XCLBIN_PATH="$HOME/.config/flm" flm run gemma4-glm:4b

Kernels

FastFlowLM's NPU kernels (xclbins) are closed source and are not shipped in this repository. flm-add.py links the kernels of the official gemma4-it:e4b model (Gemma4-E4B-IT-NPU2), because this model shares the same engine family (gemma4e) and architecture.

Model

Registry tag: gemma4-glm:4b
Engine family: gemma4e
Kernel source: Gemma4-E4B-IT-NPU2
Context length: 131,072 tokens (from config)
model.q4nx size: 7.14 GB
Base model: google/gemma-4-E4B-it
License: apache-2.0

GhostWriter Influence Test (Arbitrary but repeatable benchmark)

Tested on an AMD Ryzen AI 340 Framework 13 laptop.

Metric	Value
Prompt Tokens	9,210
Completion Tokens	1,741
Total Tokens	11,006
Active KV Tokens	11,006
Max KV Token Capacity	32,768
KV Token Occupancy	33.59%
Load Duration	0.000000581 seconds
Prefill Duration (TTFT)	21.48 ms
Decoding Duration	216.73 ms
Prefill Speed	431.36 tokens/sec
Decoding Speed	8.03 tokens/sec

Original model card

See the upstream model card for training details, benchmarks, and upstream usage. This repository only contains the Q4NX conversion for FastFlowLM.

Upstream card: google/gemma-4-E4B-it