Abliterated version of
moonshotai/Moonlight-16B-A3B-Instruct with reduced refusals using MoE gate abliteration.
Benchmarks run on 2x RTX 4090 GPUs to verify capability preservation after abliteration.
1from transformers import AutoModelForCausalLM, AutoTokenizer
2import torch
3
4model = AutoModelForCausalLM.from_pretrained(
5 "rawcell/Moonlight-16B-A3B-Instruct-bruno",
6 torch_dtype=torch.float16,
7 device_map="auto",
8 trust_remote_code=True
9)
10tokenizer = AutoTokenizer.from_pretrained(
11 "rawcell/Moonlight-16B-A3B-Instruct-bruno",
12 trust_remote_code=True
13)
14
15messages = [{"role": "user", "content": "Your prompt here"}]
16prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
17inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
18outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
19print(tokenizer.decode(outputs[0], skip_special_tokens=True))
This model has been modified to reduce refusals. Use responsibly and in accordance with applicable laws and ethical guidelines. The creators are not responsible for misuse.
