Property	Value
Original Model	upstage/Solar-Open-100B
Original Parameters	9.46B
Pruned Parameters	3.09B
Compression Ratio	0.3261 (3.1x smaller)
Strategy	Layer + Head + FFN (Recommended)
Importance Metric	magnitude
Layers Removed	31
FFN Reduction	60.00%

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("llaa33219/Solar-Open-100B-pruned-5pct")
tokenizer = AutoTokenizer.from_pretrained("llaa33219/Solar-Open-100B-pruned-5pct")

Notes

This model was created using structured pruning techniques including:

Layer pruning (removing entire transformer layers)
Attention head pruning (reducing number of attention heads)
FFN dimension pruning (reducing intermediate layer sizes)

The pruning was based on magnitude importance scoring to preserve the most important weights.

llaa33219/Solar-Open-100B-pruned-5pct-test1

Solar-Open-100B-pruned-5pct

Pruning Details

Usage

Notes

llaa33219/
Solar-Open-100B-pruned-5pct-test1