Parameter	Value
direction_index	12.73
attn.o_proj.max_weight	1.49
attn.o_proj.max_weight_position	9.36
attn.o_proj.min_weight	0.65
attn.o_proj.min_weight_distance	4.53
mlp.down_proj.max_weight	1.14
mlp.down_proj.max_weight_position	12.76
mlp.down_proj.min_weight	1.02
mlp.down_proj.min_weight_distance	4.68

Performance

Developed by: joelewing
License: Llama 3.2 Community License Agreement
Finetuned from model: unsloth/llama-3.2-1b-instruct-bnb-4bit

Metric	This model	Original model (joelewing/Llama-3.2-1B-Instruct-Capybara)
KL divergence	0.03	0 (by definition)
Refusals	3/100	77/100

Llama-3.2-1B-Instruct-Capybara

This model is a finetune of Llama 3.2 1B on the Capybara dataset.

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.