You can deploy this model and start earning money today!

RichardErkhov/
mlabonne_-_NeuralLlama-3-8B-Instruct-abliterated-gguf

Name: mlabonne_-_NeuralLlama-3-8B-Instruct-abliterated-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

NeuralLlama-3-8B-Instruct-abliterated - GGUF

Model creator: https://huggingface.co/mlabonne/
Original model: https://huggingface.co/mlabonne/NeuralLlama-3-8B-Instruct-abliterated/

Name	Quant method	Size
NeuralLlama-3-8B-Instruct-abliterated.Q2_K.gguf	Q2_K	2.96GB
NeuralLlama-3-8B-Instruct-abliterated.IQ3_XS.gguf	IQ3_XS	3.28GB
NeuralLlama-3-8B-Instruct-abliterated.IQ3_S.gguf	IQ3_S	3.43GB
NeuralLlama-3-8B-Instruct-abliterated.Q3_K_S.gguf	Q3_K_S	3.41GB
NeuralLlama-3-8B-Instruct-abliterated.IQ3_M.gguf	IQ3_M	3.52GB
NeuralLlama-3-8B-Instruct-abliterated.Q3_K.gguf	Q3_K	3.74GB
NeuralLlama-3-8B-Instruct-abliterated.Q3_K_M.gguf	Q3_K_M	3.74GB
NeuralLlama-3-8B-Instruct-abliterated.Q3_K_L.gguf	Q3_K_L	4.03GB
NeuralLlama-3-8B-Instruct-abliterated.IQ4_XS.gguf	IQ4_XS	4.18GB
NeuralLlama-3-8B-Instruct-abliterated.Q4_0.gguf	Q4_0	4.34GB
NeuralLlama-3-8B-Instruct-abliterated.IQ4_NL.gguf	IQ4_NL	4.38GB
NeuralLlama-3-8B-Instruct-abliterated.Q4_K_S.gguf	Q4_K_S	4.37GB
NeuralLlama-3-8B-Instruct-abliterated.Q4_K.gguf	Q4_K	4.58GB
NeuralLlama-3-8B-Instruct-abliterated.Q4_K_M.gguf	Q4_K_M	4.58GB
NeuralLlama-3-8B-Instruct-abliterated.Q4_1.gguf	Q4_1	4.78GB
NeuralLlama-3-8B-Instruct-abliterated.Q5_0.gguf	Q5_0	5.21GB
NeuralLlama-3-8B-Instruct-abliterated.Q5_K_S.gguf	Q5_K_S	5.21GB
NeuralLlama-3-8B-Instruct-abliterated.Q5_K.gguf	Q5_K	5.34GB
NeuralLlama-3-8B-Instruct-abliterated.Q5_K_M.gguf	Q5_K_M	5.34GB
NeuralLlama-3-8B-Instruct-abliterated.Q5_1.gguf	Q5_1	5.65GB
NeuralLlama-3-8B-Instruct-abliterated.Q6_K.gguf	Q6_K	6.14GB
NeuralLlama-3-8B-Instruct-abliterated.Q8_0.gguf	Q8_0	7.95GB

Original model description:

license: other datasets:

mlabonne/orpo-dpo-mix-40k tags:
abliterated

Llama-3-8B-Instruct-abliterated-dpomix

This model is an experimental DPO fine-tune of an abliterated Llama 3 8B Instruct model on the full mlabonne/orpo-dpo-mix-40k dataset. It improves Llama 3 8B Instruct's performance while being uncensored.

🔎 Applications

This is an uncensored model. You can use it for any application that doesn't require alignment, like role-playing.

Tested on LM Studio using the "Llama 3" preset.

⚡ Quantization

GGUF: https://huggingface.co/mlabonne/Llama-3-8B-Instruct-abliterated-dpomix-GGUF

🏆 Evaluation

Open LLM Leaderboard

This model improves the performance of the abliterated source model and recovers the MMLU that was lost in the abliteration process.

Nous

Model	Average	AGIEval	GPT4All	TruthfulQA	Bigbench
mlabonne/Llama-3-8B-Instruct-abliterated-dpomix 📄	52.26	41.6	69.95	54.22	43.26
meta-llama/Meta-Llama-3-8B-Instruct 📄	51.34	41.22	69.86	51.65	42.64
failspy/Meta-Llama-3-8B-Instruct-abliterated-v3 📄	51.21	40.23	69.5	52.44	42.69
abacusai/Llama-3-Smaug-8B 📄	49.65	37.15	69.12	51.66	40.67
mlabonne/OrpoLlama-3-8B 📄	48.63	34.17	70.59	52.39	37.36
meta-llama/Meta-Llama-3-8B 📄	45.42	31.1	69.95	43.91	36.7

💻 Usage

python

!pip install -qU transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "mlabonne/Llama-3-8B-Instruct-abliterated-dpomix"
messages = [{"role": "user", "content": "What is a large language model?"}]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])

RichardErkhov/mlabonne_-_NeuralLlama-3-8B-Instruct-abliterated-gguf