This model is a pruned and finetuned version of meta-llama/Llama-2-13b-hf, retaining approximately 80% of parameters while maintaining strong performance through genetic algorithm pruning and RMSNorm fine-tuning.
Model Details
Base Model: meta-llama/Llama-2-13b-hf
Parameter Retention: ~80%
Pruning Method: Genetic Algorithm
Fine-tuning Method: RMSNorm calibration
Performance
Metric
Value
PPL (Before Fine-tuning)
6.76
PPL (After Fine-tuning)
5.64
Improvement
16.65%
Performance Comparison
Model
PPL (After FT)
50% params
10.03
70% params
6.59
80% params
5.64
90% params
5.01
Files Included
: Full model state dict
: This documentation
License
Llama 2 Community License (inherited from base model)
