Property	Value
Original Model	upstage/Solar-Open-100B
Original Parameters	26.87B
Pruned Parameters	4.40B
Compression Ratio	0.1639 (6.1x smaller)
Strategy	Layer + Head + FFN (Recommended)
Importance Metric	magnitude
Layers Removed	36
FFN Reduction	0.00%

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("llaa33219/Solar-Open-100B-pruned-5pct")
tokenizer = AutoTokenizer.from_pretrained("llaa33219/Solar-Open-100B-pruned-5pct")

Notes

This model was created using structured pruning techniques including:

Layer pruning (removing entire transformer layers)
Attention head pruning (reducing number of attention heads)
FFN dimension pruning (reducing intermediate layer sizes)

The pruning was based on magnitude importance scoring to preserve the most important weights.

llaa33219/Solar-Open-100B-to-4B-test-2

Solar-Open-100B-pruned-5pct

Pruning Details

Usage

Notes

llaa33219/
Solar-Open-100B-to-4B-test-2