This model is a pruned and finetuned version of meta-llama/Llama-2-13b-hf, retaining approximately 90% of parameters while maintaining strong performance through genetic algorithm pruning and RMSNorm fine-tuning.
Model Details
Base Model: meta-llama/Llama-2-13b-hf
Parameter Retention: ~90%
Pruning Method: Genetic Algorithm
Fine-tuning Method: RMSNorm calibration
Performance
Metric
Value
PPL (Before Fine-tuning)
5.80
PPL (After Fine-tuning)
5.01
Improvement
13.64%
Performance Comparison
Model
PPL (After FT)
50% params
10.03
70% params
6.59
80% params
5.64
90% params
5.01
Files Included
: Full model state dict
: This documentation
License
Llama 2 Community License (inherited from base model)
