You can deploy this model and start earning money today!

audreyt/
Brumby-14B-Base-GGUF

Name: Brumby-14B-Base-GGUF
Availability: InStock

Views

No views yet

Brumby 14B Base — GGUF

GGUF quantizations of Manifest AI's Brumby 14B Base, a 14B parameter language model using power retention — a novel attention mechanism that replaces softmax attention with a degree-2 symmetric power feature map and linear recurrence.

What is Power Retention?

Power retention (paper) is a recurrent attention mechanism that:

Expands keys and queries via a symmetric power feature map phi(x) (degree-2 polynomial expansion, head_dim → D=9216 for head_dim=128)
Maintains a recurrent state S (updated as S = decay * S + outer(phi(k), v)) with a normalizer s
Produces output as (phi(q) @ S) / (phi(q) @ s) — normalized linear attention

This gives the model constant-time per-token inference (like RWKV/Mamba) while maintaining strong language modeling performance.

Quantizations

File	Quant	Size	Description
`Brumby-14B-Base.Q2_K.gguf`	Q2_K	5.4 GB	Smallest, lower quality
`Brumby-14B-Base.Q3_K_M.gguf`	Q3_K_M	6.9 GB	Small, good balance
`Brumby-14B-Base.Q4_K_M.gguf`	Q4_K_M	8.4 GB	Recommended
`Brumby-14B-Base.Q5_K_M.gguf`	Q5_K_M	9.8 GB	Good quality
`Brumby-14B-Base.Q6_K.gguf`	Q6_K	12 GB	Very good quality
`Brumby-14B-Base.Q8_0.gguf`	Q8_0	15 GB	Near-lossless

How to Run

Important: This model requires a custom llama.cpp build with power retention support.

1. Build llama.cpp with power retention

bash

git clone https://github.com/audreyt/llama.cpp
cd llama.cpp
git checkout power-retention
cmake -B build -DGGML_CUDA=ON  # or without CUDA for CPU-only
cmake --build build -j

2. Run inference