GGUF quantized version of Morpheus v3, a 91M-parameter Mamba-2 language model for Basque (Euskara) text autocompletion with Fill-in-the-Middle (FIM) support — the model can complete text at the cursor, not just at the end.

This is a continued-pretraining of the AR-only v2 model (step 74K) on FIM-structured data.

What's new in v3 (FIM)

Capability	v2 (AR-only)	v3 (FIM)
Next-word / end-of-line completion	✅	✅
Fill-in-the-Middle (cursor mid-text)	❌	✅
Special tokens	4000	4004 (`<PRE>` `<SUF>` `<MID>` `<EOT>`)
Vocab (padded)	4016	4016

FIM training recipe (Phase 6 v3)

Base: v2 best.pt (AR checkpoint, step 3500 of Phase 6 v2)
Data: 500M tokens, 70/30 FIM/AR ratio (BigCode-style token-level splitting)
EOT loss weighting: 5× on the <EOT> stop token — forces the model to learn when to stop generating infill, not just what to generate
Packed dataset: Greedy packing of whole examples into 1025-token windows (no cross-example contamination)
LR: 1.0e-3, cosine decay, ~3,815 steps
Tokenizer: basque_unigram_fim.model (original 4000 pieces + 4 FIM tokens)

The 70/30 ratio + 5× EOT weight was selected after a senior ML engineer review of the v2 failure mode (unreliable <EOT> emission → over-generation). See Option D in the analysis below.

Available Files

File	Quant	Size	Use
`v3_fim.Q5_K_M.gguf`	Q5_K_M	64 MB	Recommended — FIM + AR
`morpheus-v2-mamba.Q4_K_M.gguf`	Q4_K_M	53 MB	AR-only (legacy v2)
`morpheus-v2-mamba.Q5_K_M.gguf`	Q5_K_M	64 MB	AR-only (legacy v2)

FIM Evaluation Results

Trained-for-behavior eval on 147 held-out FIM examples (token-level splits, 20% at linguistic boundaries):

Metric	v2 (50/50, no EOT weight)	v3 (70/30, 5× EOT weight)	Change
EOT emission rate	76.9%	88.4%	+11.5pp
Keystrokes saved	-25.2%	-5.9%	4× better
Exact match	2.7%	6.8%	2.5×
Char accuracy	31.2%	32.3%	+1.1pp
Gen length (ref=45.0)	56.8 (over-gen)	40.3	closer to target
Prefix truncation	0.0%	1.4%	minimal
└ long-bucket truncation	0.0%	2.3%	well under 15% threshold

Key findings:

The 5× EOT loss weighting worked as designed: EOT reliability up 77% → 88%, and the model stopped over-generating (56.8 → 40.3 chars vs 45.0 target).
The feared failure mode — premature truncation from over-weighting EOT — did not materialize (1.4% overall, 2.3% in the long bucket; failure threshold was 15%).
AR perplexity is stable (7.4 → 7.5) — "FIM-for-free" property holds even with 70% FIM data.

Model Details

Architecture: Mamba-2 (State Space Model), 24 layers, d_model 768
Parameters: 91M
Checkpoint: Phase 6 v3 best.pt (step 3500)
AR perplexity: 7.5 | FIM perplexity: 7.9
Tokenizer: basque_unigram_fim.model — 4004 pieces (original 4000 + <PRE> <SUF> <MID> <EOT>)
Trained without BOS token (add_bos_token=false)

FIM Token Format

The model uses Code Llama-style FIM tokens. To do a fill-in-the-middle completion, structure the prompt as:

<PRE>{prefix}<SUF>{suffix}<MID>

The model generates the infill and emits <EOT> when done.

Token	ID	Purpose
`<PRE>`	4000	Marks start of prefix
`<SUF>`	4001	Marks start of suffix
`<MID>`	4002	Marks start of generation (infill)
`<EOT>`	4003	End-of-infill (stop token)
4004–4015	—	Padding (unused)

Usage

With the Morpheus demo server (recommended)

The Morpheus demo includes a FastAPI proxy that handles FIM templating, token-ID encoding, and an OpenAI-compatible API:

bash

cd demo
docker compose -f docker-compose.yml -f docker-compose.local.yml up -d --build
# Open http://localhost:9090/editor.html

Set MORPHEUS_MODEL=v3_fim.Q5_K_M.gguf to use this model.

Direct llama-server (raw FIM)

bash

llama-server -m v3_fim.Q5_K_M.gguf --host 127.0.0.1 --port 8080 -ngl 0 --threads 8

# FIM completion:
curl http://localhost:8080/completion \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "<PRE>Kaixo, <SUF> moduz?<MID>",
    "n_predict": 20,
    "temperature": 0.2,
    "top_k": 5,
    "stop": ["<EOT>"]
  }'

⚠️ Critical: token-ID prompts, not strings

As with v2, deploy via token-ID prompts (encode with sentencepiece using basque_unigram_fim.model, send IDs to /completion). The proxy handles this automatically. See the v2 model card for the full explanation.

Decoding parameters (recommended)

These are wired into the demo proxy and editor:

Parameter	Value	Rationale
`temperature`	0.2	Low-but-nonzero: recovers rank-2 correct tokens that greedy misses
`top_k`	5	Small nucleus; P7 finding: 5 correct answers sit at rank 2 in top-5
`stop` (FIM)	`["<EOT>", "\n\n"]`	Model emits `<EOT>`; fallback to paragraph boundary
`repeat_penalty` (FIM)	1.0	FIM legitimately reuses context words — do not penalize

Intended Use

Desktop text-editor ghost-text autocompletion for Basque prose. The Mamba-2 architecture's O(1) decode cost makes it well-suited to long editing sessions where latency per token matters more than parallelism.

Not intended for: instruction following, chat, translation, or factual QA. This is a narrow autocomplete model.

License

Apache-2.0.

Citation

Morpheus v3 — Mamba-2 FIM Autocomplete for Basque.
itzune/morpheus-gguf, 2026.