The RWKV-MS memory adapter was built for the tau2 telecom/mobile-data tool-agent experiment. The accepted sidecar was exported from the step-100 checkpoint of the formatrefresh_continue200_len192_layers0_5_qo_r8 tau2 continuation run in the online-memory checkpoint repo:

xiaol/gemma-4-e4B-hybrid-rnn-mem-rwkv-fable5-gpt5.5-v1

Despite the historical repo name, the accepted checkpoint metadata says it was trained on deterministic tau2 telecom mobile-data rule-planner traces and a format-refresh continuation. It was not trained on GPT-5.5-generated data. The fable5_gpt55_data entry in the metadata is a planned upgrade note, not the data used for this accepted checkpoint.

The base GGUF and multimodal projector are normal llama.cpp artifacts. The RWKV-MS memory file is a GGUF sidecar containing adapter tensors and metadata. Executing the sidecar requires the patched llama.cpp fork below; stock llama.cpp can parse ordinary GGUF files but does not run this RWKV-MS memory sidecar.

Patched runtime:

text

https://github.com/xiaol/llama.cpp-online-memory
commit: 85da0c63b Add Gemma4 RWKV-MS GGUF sidecar runtime
base upstream: ggml-org/llama.cpp 1ec44d1

Files

File	Size	SHA-256
`gemma-4-E4B-it-Q8_0.gguf`	8,031,240,160 bytes	`fb8f0c032de00b18c710824af3c7e5777c71e5fb60b13f13575f0a9e92ddecd0`
`mmproj-gemma-4-E4B-it-Q8_0.gguf`	559,874,528 bytes	`51d4b7fd825e4569f746b200fccc5332bf914e8ef7cbe447272ce4fec6df3db6`
`gemma-4-E4B-it-rwkv-ms-memory.gguf`	1,663,840 bytes	`0c646a776b5b12c9d3657ffd2e5e581be1eb46e858f1f404afeaa7077c02974e`

The sidecar is bound to the exact base GGUF SHA-256 above. Loading it with a different quantization or regenerated GGUF will fail with a hash mismatch.

Run Base GGUF

bash

/path/to/llama-server \
  -m gemma-4-E4B-it-Q8_0.gguf \
  --mmproj mmproj-gemma-4-E4B-it-Q8_0.gguf \
  --alias gemma-4-e4b-it-q8 \
  --host 127.0.0.1 \
  --port 8080 \
  --ctx-size 8192 \
  --n-gpu-layers 999 \
  --jinja \
  --reasoning off

Run RWKV-MS Sidecar

Use the patched fork and the constrained one-sequence mode:

bash

/path/to/llama-server \
  -m gemma-4-E4B-it-Q8_0.gguf \
  --alias gemma-4-e4b-it-rwkv-ms-q8 \
  --host 127.0.0.1 \
  --port 18083 \
  --ctx-size 8192 \
  --batch-size 2 \
  --ubatch-size 1 \
  --n-gpu-layers 999 \
  --parallel 1 \
  --jinja \
  --reasoning off \
  --rwkv-ms-sidecar gemma-4-E4B-it-rwkv-ms-memory.gguf \
  --no-cont-batching \
  --no-context-shift \
  --no-cache-prompt \
  --no-cache-idle-slots \
  --cache-ram 0 \
  --ctx-checkpoints 0 \
  --slot-save-path ./slots

Current RWKV-MS Runtime Limits

Gemma4 E4B path only.
Sidecar must match the exact base GGUF SHA-256.
Target layers 0-5.
q and o delta hooks only.
num_state_heads=1.
rwkv_ms_num_states=4, chunk size 1024.
One active sequence/slot is the tested path.
Use --parallel 1 and --ubatch-size 1.
Continuous batching, context shift, prompt-cache reuse, speculative decoding, RAM prompt cache, and context checkpoints are disabled for this sidecar mode.
Slot save/restore is experimental and intended for slot 0 exact-prefix restore checks.
Online memory state lives inside the active runtime/session. It is not a durable cross-request memory database.

Validation Snapshot

Validation was run locally on 2026-06-26 with the patched llama.cpp CUDA build.

Runtime health:

text

ok: true
base_gguf_sha256: fb8f0c032de00b18c710824af3c7e5777c71e5fb60b13f13575f0a9e92ddecd0
sidecar_base_gguf_sha256: fb8f0c032de00b18c710824af3c7e5777c71e5fb60b13f13575f0a9e92ddecd0

Raw deterministic contrast suite:

text

temperature=0, top_k=1, top_p=1, seed=123, n_predict=64, cache_prompt=false
8/8 raw-completion prompts produced different base vs RWKV-MS continuations.

Example:

text

Prompt: Hi
Base:    ! I'm excited to chat with you. I'm here to help you with whatever you need. Just let me know how I can assist you today!
RWKV-MS: ! I'm excited to chat with you. What's on your mind today?

The chat-history prompt suite is also included in reports/. That suite mostly shows semantic equivalence because both endpoints receive the same conversation history in each request. It should not be read as proof of a memory advantage.

Tau2 training metadata from the exported sidecar:

text

base_model: google/gemma-4-E4B-it
selected_checkpoint: step-100
data: tau2 telecom mobile-data rule-planner traces plus format-refresh continuation
wrapped_layers: 0-5
delta_heads: q,o
rank: 8
alpha: 16
trainable_memory_params: 797,808
reported tau2 telecom 20-task screen: 14/20, pass_hat_1=0.70

Reports

reports/raw_completion_contrast_suite_20260626_225007.md
reports/raw_completion_contrast_suite_20260626_225007.jsonl
reports/prompt_suite_compare_20260626_215243.md
reports/prompt_suite_base_20260626_215243.jsonl
reports/prompt_suite_rwkv_ms_20260626_215243.jsonl
reports/rwkv_ms_runtime_health_20260626_215243.json
reports/rwkv_ms_memory_sidecar_manifest.json
reports/gguf_memory_manifest.json
reports/contrast_trace_20260626_224632.jsonl

Notes

This is an experimental research/runtime artifact. It is useful for inspecting GGUF sidecar loading, deterministic generation changes, RWKV-MS recurrent state serialization checks, and constrained llama.cpp integration behavior. It is not yet a production-fast or fully general llama.cpp memory backend.

xiaol/gemma-4-e4B-hybrid-rnn-mem-rwkv-fable5-gpt5.5-v1-GGUF

Gemma4 E4B Hybrid RNN Memory RWKV GGUF

Files

Run Base GGUF

Run RWKV-MS Sidecar

Current RWKV-MS Runtime Limits

Validation Snapshot

Reports

Notes

xiaol/
gemma-4-e4B-hybrid-rnn-mem-rwkv-fable5-gpt5.5-v1-GGUF