Solar-Open2-250B-GGUF
Unofficial community
GGUF quantizations of
upstage/Solar-Open2-250B.
Not affiliated with, sponsored by, or endorsed by Upstage. This is an
independent derivative work.
Available quantizations
Q4_K_M (~141 GB) — Solar-Open2-250B-Q4_K_M-*-of-*.gguf. The 128-wide
KDA gate projections (ssm_f_b / ssm_g_b) fall back q4_K→q5_0, as is
normal for Kimi-family linear-attention models.Q8_0 (~250 GB) — Solar-Open2-250B-Q8_0-*-of-*.gguf. Near-lossless; no
fallbacks (Q8_0's 32-wide blocks quantize the 128-wide gates cleanly).
Both are split into <50 GB shards; point llama.cpp at the first shard of a
set and it loads the rest automatically.
Modification notice: these files were modified from the original —
converted to GGUF and quantized from the original bf16 weights.
⚠️ Requires a patched llama.cpp
Solar-Open2 is a hybrid MoE (KDA linear attention + gated GQA + DeepSeek-V3-style
MoE) and its architecture is not in upstream llama.cpp. To run these GGUFs you
must build llama.cpp from the fork that adds the solar-open2 architecture:
⚠️ This fork is AI-created and will not be upstreamed
That fork — the HF→GGUF converter, the C++ inference graph, the reasoning-parser
patch, and these quantizations — was created by AI (Claude Opus 4.8). It has
not been numerically validated against Upstage's reference implementation, and
no pull request will be opened against upstream llama.cpp. Treat it as an
experimental, community-grade artifact, and review the code yourself before
relying on it — you are responsible for anything you run.
It has been verified to load and generate coherent, factually correct output
on a CUDA build, and reasoning is parsed into reasoning_content — but see the
caveat above.
Usage
1# reasoning model — a repeat penalty is recommended
2llama-cli -m Solar-Open2-250B-Q4_K_M-00001-of-000NN.gguf \
3 -ngl 99 -c 8192 --jinja --reasoning-format deepseek \
4 --repeat-penalty 1.2 -cnv
llama.cpp loads the shard set automatically when you point it at the first shard.
License
Built with Solar. Licensed under the
Upstage Solar License (see the
LICENSE file in this repo, included per Section 4(e)). The original
Solar-Open2 model is © Upstage Co., Ltd. Per the license, this derivative's name
begins with "Solar" and displays "Built with Solar." Use of the "Solar" name does
not imply any endorsement by or official relationship with Upstage.
