Changing the value of kv_count from 34 to 40 indicates an increase in the number of key-value pairs in the model. These key-value pairs are mainly used to represent attention information within neural networks, particularly in Transformer-type models such as LLaMA.

This is a merge of pre-trained language models created using mergekit.

This model was merged using the passthrough merge method using Sao10K/Fimbulvetr-11B-v2 as a base.

The following models were included in the merge:

The following YAML configuration was used to produce this model: