The GPT-OSS neural architecture and MXFP4 quantization are unchanged. The
modification is a Siliang-specific expert-major GGUF storage layout and
metadata extension for routed Mixture-of-Experts weights. No fine-tuning or
additional quantization pass was performed.
[!IMPORTANT]
This file is not a general-purpose GGUF. It requires Siliang Engine v0.1.1
or a compatible build that understands the siliangem.* metadata contract.
Plain upstream llama.cpp and other GGUF applications do not understand the
packed expert-major tensors.
Modification notice
This repository redistributes modified model material. The source MXFP4 GGUF's
routed-expert weights were physically rearranged into Siliang's expert-major
layout. The model architecture, expert IDs, tokenizer metadata, and intended
model behavior were not deliberately changed by the repack.
What the repack changes
Stock MoE GGUFs store each routed-expert projection as a tensor spanning all
experts. This repack instead places the projections for each expert together:
[expert 0: gate | up | down][expert 1: gate | up | down] ...
Each layer is represented by one packed region. Siliang Engine reconstructs
the original logical gate, up, and down tensors from the embedded geometry.
This lets one arena miss fetch an expert with one sector-aligned contiguous
read rather than separate scattered projection reads.
The layout is designed to reduce fragmented read submissions, reduce reliance
on Windows mmap page faults for routed-expert weights, and lower effective
expert-fetch latency. It does not guarantee lower total bytes read, fewer
system-wide page faults, or a specific speedup. Results depend on storage,
available RAM, memory pressure, routing behavior, and arena sizing.
Accelerated runtime: Windows. Linux support is not available in the current
experimental release.
Memory mapping must remain enabled. Siliang deliberately rejects this layout
with --no-mmap because its logical expert tensors are strided views.
Arena acceleration is opt-in. Without a positive SILIANGEM_CACHE_MIB,
Siliang uses its mmap path and does not reserve an arena.
GPT-OSS requires the Harmony response format. Use a chat interface or
conversation mode that applies the model's GPT-OSS chat template; raw
unformatted prompts are not equivalent.
Quickstart
Download and extract a Siliang Engine v0.1.1 Windows release. From its root,
choose an arena budget measured for your machine and leave enough memory for
Windows, non-expert weights, the KV cache, and GPU shared-memory pressure:
This artifact passes that check. A deterministic current-namespace smoke also
produced identical 32-token decoded output with the arena disabled and enabled;
the arena armed without an mmap fallback. That smoke established structural
and runtime-path correctness only. Its timings were intentionally excluded.
A structural pass does not prove semantic equivalence to the source model or
replace an application-specific correctness test.
Performance evidence
The retained GPT-OSS benchmark compares the same repacked GGUF through two
Siliang paths. It does not compare a stock GGUF with this repack.
Path
Median decode
Observed range
Repacked GGUF using mmap
1.972 tok/s
1.964-2.030 tok/s
Same GGUF using an 18 GiB deferred arena
4.052 tok/s
3.953-4.094 tok/s
This was a historical matched run with three fresh process starts per arm, a
48-token warmup, and 256 measured tokens. The median arena change was +105.5%
(2.06x). It isolates the arena and deferred direct-I/O path on one repacked
file; it does not measure the repack's benefit against stock GPT-OSS. Complete
model and runtime composite hashes were not retained for this historical run,
so it is not current release-validation evidence.
The test system used Windows 11, an AMD Ryzen 5 2600, 24 GB of RAM, an NVIDIA
GeForce RTX 2070 with 8 GB of VRAM, and a WD_BLACK SN850X NVMe SSD operating
through a PCIe 3.0 x4 platform link. The 18 GiB arena created high memory
pressure on this system. These measurements are experimental system-specific
evidence, not a general performance guarantee. Full methodology and caveats
are in the
Siliang Engine performance notes.
No retained GPT-OSS control currently demonstrates fewer total disk bytes,
fewer operating-system page faults, or the isolated stock-versus-repack
effect. No bit-for-bit whole-file equivalence claim is made.
The model material is distributed under the Apache License 2.0. The complete
license text is included in LICENSE. Review the upstream model
card for capabilities, safety evaluations, intended use, and broader model
limitations.
