Expert pruning is the process of removing a fraction of the experts from each Mixture-of-Experts (MoE) layer of a language model while keeping the rest of the architecture intact. The router, the attention blocks, the embeddings, the tokenizer, and the chat template are unchanged. Only the per-layer expert table is reduced, which directly shrinks the model's parameter count, memory footprint and (at inference) the amount of compute spent on the MoE block.

In this particular release the router weight matrix, the router bias and the expert tensors are sliced along the expert dimension using random-moe-pruning. The default configuration is used (--prune-level 0.25, --top-k 4, --num-examples 1000, --random-seed 42).

Provenance

Field	Value
Base model	`openai/gpt-oss-20b`
Original experts / layer	32
Pruned experts / layer	24
Experts kept per token	4
Prune level	0.25

The metadata fields pruned_from_num_experts and pruned_to_num_experts in config.json reflect the change for traceability.

How to use

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "valendra/gpt-oss-20b-mini"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

Caveats

This checkpoint is experimental. Quality has not been benchmarked against the base model.
The pruning is dataset-free, so the chosen experts reflect the router's behaviour on random Gaussian probes rather than on a calibration distribution. See the project README for the mathematical justification.
The same chat_template.jinja from the base model is reused.
License follows the base model (Apache 2.0).

valendra/experimental-gpt-oss-17b-mini

gpt-oss-20b-mini

What is this?

Provenance

How to use

Caveats

valendra/
experimental-gpt-oss-17b-mini