This model is a pruned and finetuned version of meta-llama/Llama-2-7b-hf, retaining approximately 70% of parameters while maintaining strong performance through genetic algorithm pruning and RMSNorm fine-tuning.
Model Details
Base Model: meta-llama/Llama-2-7b-hf
Parameter Retention: ~70%
Pruning Method: Genetic Algorithm
Fine-tuning Method: RMSNorm calibration
Performance
Metric
Value
PPL (Before Fine-tuning)
11.54
PPL (After Fine-tuning)
8.25
Improvement
28.54%
Performance Comparison
Model
PPL (After FT)
50% params
13.84
70% params
8.25
80% params
6.71
90% params
5.62
Files Included
: Full model state dict
: This documentation
License
Llama 2 Community License (inherited from base model)
