GLM-5 (5-Layer Truncated Variant)
This repository contains a heavily truncated 5-layer variant of the original THUDM GLM-5 model.
It is created by keeping only the first 5 transformer layers and removing the remaining depth of the original model.
Important Notice
This is NOT the full GLM-5 model and is not functionally equivalent to the original checkpoint.
Key differences:
- Only the first 5 transformer layers are retained
- Significant reduction in model capacity and reasoning ability
- Outputs are degraded compared to the full model
- Intended for research and experimentation only
Purpose
This model is intended for:
- Research on model depth reduction
- Latency and memory profiling experiments
- Studying early-layer representations in LLMs
- Building lightweight experimental inference pipelines
- Distillation / student model initialization
It is not recommended for production use.
Model Details
- Base model: GLM-5
- Architecture: Transformer (MoE-based in original model)
- Layers kept: 5 (layers 0-4)
- Layers removed: all higher layers
- Vocabulary / tokenizer: unchanged from original
Files
This repository includes:
config.json: modified to reflect 5 layersmodel.safetensors: truncated weights- tokenizer files: identical to original GLM-5 tokenizer
Usage
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_id = "sofiadsg/zai-glm5-5layers"
4
5tokenizer = AutoTokenizer.from_pretrained(
6 model_id,
7 trust_remote_code=True
8)
9
10model = AutoModelForCausalLM.from_pretrained(
11 model_id,
12 trust_remote_code=True,
13)
Limitations
Because most transformer layers are removed:
- Reasoning depth is heavily reduced
- Long-context coherence is degraded
- Instruction-following quality may be unstable
- Outputs may be repetitive or incomplete
This model should be treated as a structural research artifact, not a general-purpose LLM.
Origin
This model is derived from the original GLM-5 checkpoint released by zai.
No additional training was performed; this is a direct structural truncation of the base model.
License
This model inherits the license of the original GLM-5 release.
Users must comply with the original models terms of use.
Recommendation
For better performance, consider:
- Using the full GLM-5 model when possible
- Or training a distilled 5-layer student model instead of truncation
Disclaimer
This repository is provided for research purposes only.
No guarantees are made regarding correctness, stability, or performance.
