This is a merge of meta-llama/Meta-Llama-3.1-8B created using
mergekit, with respect to the paper
"The Unreasonable Ineffectiveness of the Deeper Layers"This model was merged using the passthrough merge method.
1dtype: bfloat16
2merge_method: passthrough
3slices:
4- sources:
5 - layer_range: [0, 23]
6 model: meta-llama/Meta-Llama-3.1-8B
7- sources:
8 - layer_range: [28, 32]
9 model: meta-llama/Meta-Llama-3.1-8B
Carbon emissions can be estimated using the
Machine Learning Impact calculator presented in
Lacoste et al. (2019).
