Quantization made by Richard Erkhov.
library_name: transformers
license: llama3.1
base_model: meta-llama/Meta-Llama-3.1-70B-Instruct
tags:
This is an uncensored version of
Llama 3.1 70B Instruct created with abliteration (see
this article to know more about it) using
@grimjim's recipe.
I adapted this recipe to Llama 3.1 70B using
failspy/Meta-Llama-3-70B-Instruct-abliterated-v3.5 and optimized the LoRA rank.
The model is fully uncensored in my tests and maintains a high level of quality. A more rigorous evaluation is still needed to measure the impact of this process on benchmarks.
Special thanks to
@grimjim for this technique (see his
8B model) and
@FailSpy for his
70B abliterated model. Please follow them if you're interested in abliterated models.
In addition, thanks to
brev.dev for providing me with compute!
This model was merged using the
task arithmetic merge method using ./meta-llama/Meta-Llama-3.1-70B-Instruct + Llama-3-70B-Instruct-abliterated-LORA as a base.
1base_model: meta-llama/Meta-Llama-3.1-70B-Instruct+Llama-3-70B-Instruct-abliterated-LORA
2dtype: bfloat16
3merge_method: task_arithmetic
4parameters:
5 normalize: false
6slices:
7- sources:
8 - layer_range: [0, 80]
9 model: meta-llama/Meta-Llama-3.1-70B-Instruct+Llama-3-70B-Instruct-abliterated-LORA
10 parameters:
11 weight: 1.0
1# Setup
2git clone https://github.com/arcee-ai/mergekit.git
3cd mergekit && pip install -e .
4pip install bitsandbytes
5
6# Extraction
7mergekit-extract-lora failspy/Meta-Llama-3-70B-Instruct-abliterated-v3.5 meta-llama/Meta-Llama-3-70B-Instruct Llama-3-70B-Instruct-abliterated-LORA --rank=64
8
9# Merge using previous config
10mergekit-yaml config.yaml Llama-3.1-70B-Instruct-lorablated --allow-crimes --lora-merge-cache=./cache
