You can deploy this model and start earning money today!

RichardErkhov/
oopere_-_pruned10-llama-3.2-3B-gguf

Name: oopere_-_pruned10-llama-3.2-3B-gguf
Availability: InStock

endpoints_compatible

gguf

template

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

pruned10-llama-3.2-3B - GGUF

Model creator: https://huggingface.co/oopere/
Original model: https://huggingface.co/oopere/pruned10-llama-3.2-3B/

Name	Quant method	Size
pruned10-llama-3.2-3B.Q2_K.gguf	Q2_K	2.13GB
pruned10-llama-3.2-3B.IQ3_XS.gguf	IQ3_XS	2.22GB
pruned10-llama-3.2-3B.IQ3_S.gguf	IQ3_S	2.28GB
pruned10-llama-3.2-3B.Q3_K_S.gguf	Q3_K_S	2.28GB
pruned10-llama-3.2-3B.IQ3_M.gguf	IQ3_M	2.32GB
pruned10-llama-3.2-3B.Q3_K.gguf	Q3_K	2.32GB
pruned10-llama-3.2-3B.Q3_K_M.gguf	Q3_K_M	2.32GB
pruned10-llama-3.2-3B.Q3_K_L.gguf	Q3_K_L	2.36GB
pruned10-llama-3.2-3B.IQ4_XS.gguf	IQ4_XS	2.47GB
pruned10-llama-3.2-3B.Q4_0.gguf	Q4_0	0.32GB
pruned10-llama-3.2-3B.IQ4_NL.gguf	IQ4_NL	0.56GB
pruned10-llama-3.2-3B.Q4_K_S.gguf	Q4_K_S	2.53GB
pruned10-llama-3.2-3B.Q4_K.gguf	Q4_K	2.53GB
pruned10-llama-3.2-3B.Q4_K_M.gguf	Q4_K_M	2.53GB
pruned10-llama-3.2-3B.Q4_1.gguf	Q4_1	0.32GB
pruned10-llama-3.2-3B.Q5_0.gguf	Q5_0	0.32GB
pruned10-llama-3.2-3B.Q5_K_S.gguf	Q5_K_S	2.75GB
pruned10-llama-3.2-3B.Q5_K.gguf	Q5_K	2.76GB
pruned10-llama-3.2-3B.Q5_K_M.gguf	Q5_K_M	2.76GB
pruned10-llama-3.2-3B.Q5_1.gguf	Q5_1	0.33GB
pruned10-llama-3.2-3B.Q6_K.gguf	Q6_K	3.0GB
pruned10-llama-3.2-3B.Q8_0.gguf	Q8_0	0.42GB

Original model description:

library_name: transformers license: llama3.2 metrics:

accuracy
perplexity base_model:
meta-llama/Llama-3.2-3B

Model Card for oopere/pruned20-llama-1b

This model is a pruned version of the Llama-3.2-3B model, with a parameter reduction of 10% in the MLP Layers. The pruning process aims to enhance computational efficiency while maintaining acceptable performance across specific tasks. This model is not intended to be used directly, but rather to be fine-tuned for specific tasks where it can achieve equal or superior performance compared to fine-tuning the base model for the same task.

Model Details

Model Type: Pruned version of LLaMA-3.2 using structured pruning
Original Model: meta-llama/Llama-3.2-3B
Pruning Method: Structured pruning of MLP layers using importance scores based on absolute maximum weights
Size Reduction: 7,47% (from 3.21B to 3B parameters)
Architecture: Same as original LLaMA but with reduced MLP layer sizes
Language(s): Same as original model
License: Same as original model
Developed by: Pere Martra

These models are part of the study "Exploring GLU Expansion Ratios: Structured Pruning in Llama-3.2 Models". They explore structured pruning in GLU-based architectures using Llama-3.2 (1B and 3B variants). The pruning experiments target optimal expansion ratios to balance performance, computational efficiency, and environmental sustainability. The models were evaluated across multiple benchmarks, including BoolQ, ARC-Easy, and MUSR, and demonstrate significant efficiency gains while maintaining robust task performance.

Performance on Standard Benchmarks

Benchmark	Original Model	Pruned Model	Relative Change
ARC-Easy	65.19%	60.69%	-6.9%
BoolQ	64.16%	51.22%	-20.2%
LAMBADA-OpenAI	62.20%	59.64%	-4.1%
LAMBADA-Standard	53.46%	54.61%	+2.2%

Key Findings

Surprisingly, an improvement is observed on the LAMBADA-Standard benchmark, with a 2.2% relative increase in accuracy.
Maintains competitive performance on binary classification tasks (BoolQ), with a 20.2% relative decrease in accuracy.
Moderate degradation observed on reasoning tasks (ARC-Easy), with a 6.9% relative decrease in accuracy.
Minimal impact on long-range comprehension (LAMBADA-OpenAI), with only a 4.1% relative decrease in accuracy.

Limitations

Reduced performance on tasks requiring complex reasoning, with moderate degradation observed on benchmarks like ARC-Easy.
Noticeable decrease in accuracy on binary classification tasks, as seen in BoolQ.
Mixed results on long-range dependencies, with minimal degradation on LAMBADA-OpenAI but variability across benchmarks.
May not be suitable for applications requiring consistently high accuracy across diverse language tasks.

Implementation Details

Pruning Notebook: Detailed implementation and methodology
GitHub Repository: LLM Course

Pruning Method

Technique: Structured pruning targeting MLP layers
Pruning Ratio: 10% of neurons removed from MLP layers
Selection Criteria: Importance scoring based on absolute maximum weights
Architecture Specifics: Maintained GLU structure during pruning

Hardware Requirements

Reduced memory footprint compared to original model
Can run on hardware with ~10% less memory than original

Acknowledgments

Thanks to Mariusz Kurman for creating llama-pruning, a library that extends and improve this pruning methodology.

RichardErkhov/oopere_-_pruned10-llama-3.2-3B-gguf