You can deploy this model and start earning money today!

pinkelephantlimited/
pinkelephant-llm-48b-s-gguf

Name: pinkelephant-llm-48b-s-gguf
Availability: InStock

Views

No views yet

🐘 Pink Elephant 48B-S — GGUF

Dense-14B brain. Sparse 48B reach. Aligned to be helpful. Run it on your own machine.

From a single dense core to an SFT + DPO–aligned sparse Mixture-of-Experts flagship — converted, quantized, and Ollama-ready so you can run the entire success story locally.

License: MIT

Params: 47.7B

Active: 30.7%

Experts

Quant: Q4_K_M

Context: 16k

Pipeline: SFT+DPO

Runtime: Ollama

🚀 At a glance

Pink Elephant 48B-S is the culmination of a six-stage lineage: a dense 14B foundation, upcycled into an 8-expert sparse MoE, refined through a second training epoch (48B-S), aligned with supervised fine-tuning and direct preference optimization, then merged bit-exact and converted to GGUF — ready to run in Ollama / llama.cpp on a single consumer GPU.

What you get	Why it matters
🧠 47.7B total params — only ~14.66B active / token (30.7%)	Dense-48B quality at a sparse fraction of the compute
⚡ 3.2× fewer FLOPs vs. a dense 48B	Faster inference on the hardware you already own
🎯 8 experts, top-2 routing	Specialized, focused reasoning per token
🛡️ SFT + DPO aligned	Predictable, helpful, preference-tuned behavior
📦 Q4_K_M quantized (8.3 GB)	Fits on an 8–12 GB consumer GPU
🚀 Ollama-ready	One command from zero to running
🆓 MIT license	Free for commercial use & research

✅ Tested — corrected build runs clean

We download the corrected Q4_K_M GGUF (pe_48b_s_dpo_fix_Q4_K_M.gguf) fresh from this repo and ran it end-to-end in llama.cpp / Ollama: it offloads to GPU, and produces coherent, on-topic, non-degenerate output. (The earlier 8-expert build lost the router bias and could garble; the shipped files are the corrected 2-expert build — see docs/VERIFICATION.md and tools/conversion/pinkelephant.py.) See the 📦 GGUF files table for exactly which hardware each file needs.

Verified aligned benchmarks (bf16, greedy, single NVIDIA RTX PRO 6000 Blackwell — our own infrastructure):

GSM8K: 68.5%
MMLU (5-shot): 60.0%

Ultimate Report

How to Run

Live Demo

🚀 Start here for clean output: docs/HOW_TO_RUN.md — correct ChatML template + sampler settings. 📘 Read the whole story: docs/ULTIMATE_REPORT.md — full traceability for investors, engineers, and users.

💡 Why a sparse MoE?

Pretraining a 48B model from scratch is expensive — and wasteful, because most of that compute goes to reconstructing what a good dense 14B already knows. Pink Elephant took the smarter route:

Start with a proven dense brain. A 14B foundation, instruction-tuned on code + math, scoring HumanEval 71.34% and MATH-500 63.40%.
Upcycle, don't retrain. Split the feed-forward weights into 8 complementary experts, exactly preserving parent behavior at initialization — so you inherit the dense model's knowledge instead of relearning it.
Sparse compute, dense capacity. Only top-2 of 8 experts activate per token (~30.7%), giving you 47.7B parameters of knowledge with 3.2× fewer FLOPs than a dense 48B.

The result: more capability per flop, per watt, per dollar — on hardware you can actually afford to own.

🏆 The Success Journey

One lineage, six milestones: a proven dense core → upcycled to sparse scale → refined → supervised fine-tuned → preference-aligned → bit-exact merged and converted to GGUF.

🔄 The Lineage

🎬 The Story, Act by Act

Act I — The Dense Core: 🐘 Pink Elephant 14B

pink-elephant-llm-14b

A dense 14.66B decoder-only model, instruction-tuned on 99,661 curated code + mathematics examples (bigcode/self-oss-instruct + open-r1/OpenR1-Math). It proved the thesis:

HumanEval pass@1: 71.34%
MATH-500: 63.40%

Act II — The Upcycle: Dense 14B → Sparse 48B MoE

pinkelephant-llm-48b

Rather than train 48B from scratch, Pink Elephant upcycled the 14B foundation into an 8-expert Mixture-of-Experts — feedforward weights split and replicated into eight complementary experts per layer, exactly preserving the parent's behavior at initialization. A sparse brain with dense-scale capacity at ~30.7% activation per token.

Act III — The Refinement: 48B-S

pinkelephant-llm-48b-s

A second full epoch (3,114 steps) on the same curriculum with a freshly restarted cosine schedule. Training loss fell 32% (0.0616 → 0.0419) with validation loss steady near 0.50 — refinement, not overfitting.

Act IV — Supervised Fine-Tuning (SFT)

pinkelephant-llm-48b-s-sft

An industrial-standard QLoRA recipe (frozen 4-bit NF4 base purely as a memory-efficient training vehicle; all math in native bf16; adapters full precision) fine-tuned the whole 47.7B on a single GPU. Training loss fell 0.65 → ~0.44; held-out CE identified step 90,000 as the best adapter (CE 0.5392).


SFT training loss	SFT held-out evaluation → best at step 90k

Act V — Direct Preference Optimization (DPO)

pinkelephant-llm-48b-s-dpo

An alignment campaign using a frozen reference + trainable policy (QLoRA-for-DPO). The defining engineering decision: an evidence-based turning-point early stop — we watched the loss curve, found its true minimum 0.4763 at step 11,579, then stopped 1,500 steps later. A clean 2,000-pair held-out preference set shows ~0.65 preference accuracy.

DPO training loss with min-loss turning point

Act VI — The Merge & Conversion (This Repo) 🚀

pinkelephant-llm-48b-s-gguf — you are here

We fused the best SFT adapter (step 90,000) + best DPO adapter (min-loss step 11,579) into the 48B-S base — bit-exact — then converted the non-standard fused-layout MoE to standard GGUF and quantized it.

base → moe → train → sft → dpo → verification → release

🎯 Use cases

Because it is a compact, aligned, MIT-licensed sparse MoE that fits on consumer hardware, Pink Elephant 48B-S GGUF is ideal for:

Scenario	Why it fits
🧑‍💻 Local code & math copilot	Trained on code + math; strong HumanEval / GSM8K; runs offline
🔒 Private / on-prem deployment	8.3 GB model — no data leaves your machine
🏭 Edge & cost-sensitive serving	3.2× fewer FLOPs than dense 48B; 30.7% activation saves power/cost
📚 Research & experimentation	MIT license, fully reproducible lineage & verification
🎓 Education / demos	One-command Ollama bring-up; the whole story is public & measured

📌 Specifications

Specification	Value
Total parameters	47,691,290,048 (47.7B)
Active params / token	~14.66B (30.7%)
Sparse compute advantage	3.2× fewer FLOPs vs. dense 48B
Architecture	MoE · 8 experts · top-2 routing
Transformer layers / hidden / expert	40 · 5,120 / 8,960
Attention heads	40 (10 KV heads, head dim 128)
Context window	16,384 tokens
Vocabulary	100,352
Source precision	Native bfloat16
Merge formula	`W = W_base + 2·(SFT_B@SFT_A) + 2·(DPO_B@DPO_A)` (r=32, α=64)
License	MIT — free for commercial use

📦 GGUF files — pick the one that fits your machine

✅ Tested (corrected build): the Q4_K_M file below (pe_48b_s_dpo_fix_Q4_K_M.gguf) was downloaded fresh from this repo and run end-to-end in llama.cpp / Ollama — created as a model, offloaded to GPU, and produced coherent, non-degenerate output. It is the corrected 2-expert build (the earlier 8-expert build dropped the router bias and could garble; superseded). See docs/VERIFICATION.md and tools/conversion/pinkelephant.py for the fix.

File	Quant	Size	Minimum hardware to run it	Best experience
`pe_48b_s_dpo_fix_Q4_K_M.gguf`	Q4_K_M	8.3 GB	≥ 16 GB RAM (CPU-only) or GPU with ≥ 8 GB VRAM for full offload	8–12 GB GPU (full offload) for interactive speed; the recommended file for Ollama
`pe_48b_s_dpo_fix.gguf`	f16 (BF16)	29.3 GB	≥ 32 GB unified memory / VRAM (e.g. 32 GB or 2×16 GB GPU)	Full-precision reference for servers; needs a 32 GB-class host

How to read this:

Choose Q4_K_M to run on a typical modern machine. It's the file we verified consistently.
- 8–12 GB GPU → full GPU offload, interactive speed. Best choice for most users.
- No GPU / 16 GB RAM → runs on CPU, slower but works.
Choose f16 only if you have a 32 GB-class unified host; otherwise use Q4_K_M.

Note on quality/speed and size: these are the corrected 2-expert (effective) build of the SFT+DPO-aligned 48B-S. It ships only the live experts {0,4} (the 8-expert source has 6 dead experts that were never routed-to), which is why the files are far smaller than the old 8-expert GGUF. Q4_K_M trades a small amount of precision for a much smaller file; f16 preserves full fidelity. Both fix the router-bias bug from the earlier release.

🚀 Run it with Ollama

Prerequisite

Install Ollama from https://ollama.com and confirm it's running:

ollama --version

Option A — Pull directly from Hugging Face

ollama pull hf.co/pinkelephantlimited/pinkelephant-llm-48b-s-gguf:Q4_K_M

Option B — Build from the bundled Modelfile

The repo ships a ready-made Modelfile with the correct ChatML template and safe sampler settings (this matters — see ⚙️ Sampler & format notes below):

dockerfile

FROM ./pe_48b_s_dpo_fix_Q4_K_M.gguf

# Lower temperature + repeat_penalty 1.0 avoid degenerate/garbled output on this MoE.
PARAMETER temperature 0.3
PARAMETER repeat_penalty 1.0
PARAMETER top_p 0.9
PARAMETER num_ctx 4096

# ChatML (OpenHermes-2.5) template — REQUIRED for correct output.
TEMPLATE """{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{- end }}<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

Download pe_48b_s_dpo_fix_Q4_K_M.gguf from the Files tab into this folder, then:

bash

ollama create pe-48b -f Modelfile
ollama run pe-48b "Write a Python function that checks if a number is prime."

Chat via the API

bash

curl http://localhost:11434/api/chat -d '{
  "model": "pe-48b",
  "messages": [{"role": "user", "content": "What is the capital of France?"}],
  "stream": false
}'

Python:

python

import requests
r = requests.post("http://localhost:11434/api/chat", json={
    "model": "pe-48b",
    "messages": [{"role": "user", "content": "Explain MoE in one sentence."}],
    "stream": False,
})
print(r.json()["message"]["content"])

⚙️ Sampler & format notes (read this before you run)

This model is ChatML / OpenHermes-2.5 aligned (SFT + DPO). Getting clean output depends on two things: the chat template, and conservative sampling.

It must be prompted in ChatML form. The bundled Modelfile already sets the correct ChatML TEMPLATE, so plain ollama run pe-48b "..." and /api/chat are handled for you. If you bypass the template and send a bare/raw prompt (e.g. via /api/generate), you must wrap it yourself, exactly like this (note the closing assistant tag):

<|im_start|>user\nWhat is the capital of France?<|im_end|>\n<|im_start|>assistant\n

A bare prompt such as "The capital of France is" will not auto-complete correctly.
Use a low temperature and keep repeat_penalty at 1.0. The default Ollama sampler (temperature ~0.7, repeat_penalty ~1.1) makes this model garble / repeat words. The bundled Modelfile pins temperature 0.3, repeat_penalty 1.0, top_p 0.9, which gives stable, non-degenerate output. For the most deterministic results you may set temperature 0. Do not raise temperature much above 0.3 without re-testing.
Known robustness note (verified). The raw-completion path with an explicit ChatML prompt (point 1) is the most reliable on this custom MoE — it produced the model's exact correct answer deterministically. The Ollama /api/chat template path generally works, but in some Ollama versions it can be less stable for this architecture; if you ever see odd output via /api/chat, switch to the raw-completion form in point 1 with "temperature": 0 (deterministic) as a reliable fallback.

Tip: prefer the chat template (or the explicit ChatML raw-completion form) rather than bare completion for natural, instruction-following output from this SFT/DPO-aligned model.

🖥️ Hardware fit

Q4_K_M (8.3 GB): runs fully offloaded on an 8–12 GB GPU (or CPU with ≥ 16 GB RAM). The corrected 2-expert build is small enough for consumer-range cards.
f16 (29.3 GB): needs ~32 GB VRAM / unified memory.

🔬 Engineering → fully reproducible in this repo

Path	What it is
`docs/BUILD_REPORT.md`	Full methodology: merge, convert, quantize, verify
`tools/conversion/pinkelephant.py`	Custom converter — splits fused QKV, splits gate/up, stacks 8 experts → LLAMA arch
`tools/pe_init.patch`	One-line registry hook into llama.cpp
`tools/make_mini_model.py`	Cheap mini-model rig that proved the converter
`scripts/merge_48b.py`	Bit-exact SFT+DPO merge (`max\|diff\| = 0.0` vs. recomputed)
`scripts/verify_merge.py`	Independent merge sanity check
`scripts/run_convert.sh` / `quant_real.sh`	GGUF conversion + Q4_K_M quantization

🧪 Full verification report

See docs/VERIFICATION.md for the complete, measurement-by-measurement evidence: bit-exact merge, all-403-tensor integrity, router top-2 routing, conversion fidelity, and the HF → Ollama round-trip.

✅ Verification highlights (measured)

Bit-exact merge: merged == base + 2·SFT + 2·DPO with max|diff| = 0.000e+00.
Tensor integrity: all 403 tensors (3 global + 40×10 per-layer) with exact shapes — incl. 3D stacked expert blocks ffn_{gate,up,down}_exps[·,·,8].
Router check: top-2 selected experts identical to source (overlap 1.0).
Weights through conversion: o_proj, norms, qkv→q match source within f16 rounding.
HF → Ollama round-trip: SHA-256 of the downloaded Q4_K_M matches the artifact that generated valid answers in Ollama.

📚 The Pink Elephant Family

Stage	Repo
🐘 Dense 14B core	`pink-elephant-llm-14b`
🔀 48B MoE (epoch-1)	`pinkelephant-llm-48b`
💎 48B-S base	`pinkelephant-llm-48b-s`
🎓 SFT adapters	`pinkelephant-llm-48b-s-sft`
🛡️ DPO adapters	`pinkelephant-llm-48b-s-dpo`
🚀 GGUF (this repo)	`pinkelephant-llm-48b-s-gguf`

🎉 Get started now

bash

# Fastest path — pull straight from Hugging Face
ollama pull hf.co/pinkelephantlimited/pinkelephant-llm-48b-s-gguf:Q4_K_M
ollama run pinkelephant-llm-48b-s-gguf "Explain mixture of experts like I'm five."

Or watch the whole story come alive in the Ultimate Project Report.

License & Commercial Use

MIT License — free for commercial and research use, modification, and redistribution. Use at your own risk.

🦾 Pink Elephant Limited · Pink Elephant 48B-S GGUF · 2026 — from a 14B dense core to a runnable, aligned MoE flagship, in one lineage.

Made with ❤️ by Pink Elephant