Microsoft open-sourced it, then deleted it from HuggingFace and GitHub (verified: 404 on both). These are long-context-imatrix GGUF quants so the weights stay in your hands — the full preserved original (bf16, 8.0 GB) is at KikoCis/FastContext-1.0-4B-SFT. Own your AI.
🔍 What FastContext is (and why it's special)
FastContext isn't a chatbot — it's a repository-exploration subagent for coding agents. Your main agent (Claude Code, Copilot, Cursor, OpenHands…) delegates file discovery to it:
Main agent asks: "where is auth handled?"
FastContext fires parallel read-only tool calls — READ / GLOB / GREP — across the repo,
and returns just the file paths + line ranges you need as compact, focused context.
Your expensive frontier agent stops burning tokens crawling directories. Microsoft's (now-deleted) announcement reported ~60% fewer tokens from the main agent and +5.5% on SWE-bench — their figures, not independently reproduced here.
the safe default — more headroom for long contexts
K-quants (Q4_K_M) = solid general quants. I-quants (IQ3_M) = smaller at similar quality; they need an imatrix (we ship ours: fastcontext4b.imatrix).
What's different vs the other FastContext GGUFs: the importance matrix here is calibrated on long, multi-thousand-token sequences (LongAlign), not the usual short generic corpus — matching the 256K regime this model was built for. For AMD Strix Halo specifically, see plunderstruck's ROCmFP4 build (different target, code-weighted imatrix).
🧮 Will it fit? (RAM/VRAM cheat-sheet)
Total ≈ weights + KV-cache (KV grows with context):
you have
quant
context you can run
4 GB
IQ3_M
~8–16K
6 GB
IQ3_M / Q4_K_M
~32K
8 GB
Q4_K_M
~64–128K
12 GB+
Q4_K_M
up to 256K native
🚀 How to run it
bash
1# llama.cpp — point it at your repo dump, ask for locations:2llama-cli -m fastcontext4b.Q4_K_M.imx.gguf -c 32768\3 -p "…repo contents…\n\nWhere is authentication handled? Return file:line ranges only."45# llama-server (use it as a subagent endpoint for your main coding agent):6llama-server -m fastcontext4b.Q4_K_M.imx.gguf -c 65536 --port 809178# Ollama (Modelfile included, 32K default):9ollama create fastcontext -f Modelfile && ollama run fastcontext
Recommended sampling: temperature 0.6, top_p 0.9, top_k 20. For pure retrieval calls, temperature 0 works well.
Subagent pattern: keep FastContext resident on a cheap local endpoint; have your main agent call it for "where is X?" queries and inject only the returned ranges into its own context.
📊 Validation — measured on these files (honest)
Needle-in-haystack retrieval (find an inserted fact inside real long documents), greedy decoding:
model
needle retrieval @~5K ctx
original (bf16)
30/30
Q4_K_M (imx)
30/30
IQ3_M (imx)
30/30
At 5K context all three — including the aggressive IQ3_M — match the original bf16 perfectly: quantization is lossless for retrieval here. Deeper long-context numbers will be added once measured on a clean harness — no placeholder claims.
Harness: llama-server + OpenAI-compat API, temp 0, 30 tasks, haystacks built from real LongAlign documents, deterministic gold.
It's a scout, not a solver — pair it with your main coding agent; don't expect it to write the patch itself.
The original repo is gone, so upstream docs/issues are gone with it; the harness conventions above are from the model's own announcement and community usage.
🗒️ Changelog
2026-07-02 v1 — IQ3_M + Q4_K_M with long-context imatrix; retrieval validated @5K (30/30 all); imatrix + Modelfile included; original preserved in the sibling repo.
