An uncensored 8B parameter language model built on Qwen3-8B, fine-tuned on 1.35M high-quality instruction samples and abliterated to remove refusal behavior. Developed for TRC (TPU Research Cloud) research.

Model Details

Architecture: Qwen3ForCausalLM (36 layers, 4096 hidden, 32 attn heads, 8 KV heads)
Parameters: 8.2B
Context Length: 4096 (trained), 40960 (max supported)
Precision: bfloat16
License: Apache 2.0

Training

SFT Phase

Base model: Qwen/Qwen3-8B
Hardware: Google Cloud TPU v6e-16 (spot)
Framework: MaxText (JAX)
Steps: 130,000 (~3 epochs)
Learning rate: 5e-6 with cosine decay
Warmup: 200 steps
Effective batch size: 16
Sequence length: 4096

Training Dataset (1.35M samples)

Dataset	Samples	Purpose
NousResearch/Hermes-3-Dataset	~959K	Core uncensored assistant behavior
allenai/tulu-3-sft-mixture	~200K	Diverse instruction following
HuggingFaceTB/smoltalk (magpie-ultra)	~100K	High quality diverse tasks
HuggingFaceTB/smoltalk (numina-cot)	~50K	Math reasoning
HuggingFaceTB/smoltalk (self-oss-instruct)	~50K	Code generation
LDJnr/Capybara	~16K	Multi-turn conversations

All data was filtered to remove refusal patterns, safety-alignment subsets, and <think> reasoning tags.

Abliteration Phase

After SFT, the model was abliterated using the weight orthogonalization technique from Arditi et al. (2024) to remove residual refusal behavior.

Technique: Multi-direction abliteration (weight orthogonalization)
Directions removed: 5
Target layers: 35, 34, 36, 33, 16 (selected by highest refusal direction scores)
Samples used: 256 harmful/harmless instruction pairs
Method: For each selected layer, the refusal direction was identified via mean difference between harmful and harmless activations, then orthogonalized out of the weight matrices.

Benchmark Results

Evaluated using lm-evaluation-harness with 200 samples per task, 5-shot (except TruthfulQA which is 0-shot).

Benchmark	Metric	Score
ARC-Challenge	acc	56.5%
ARC-Challenge	acc_norm	54.0%
HellaSwag	acc_norm	64.5%
TruthfulQA MC2	acc	48.8%
Winogrande	acc	57.0%

GGUF Quantizations

File	Quant	Size	Description
`dolphin-v2-8b-abliterated-Q8_0.gguf`	Q8_0	8.3 GB	Best quality quantization
`dolphin-v2-8b-abliterated-Q4_K_M.gguf`	Q4_K_M	4.8 GB	Good balance of quality and size

Usage with llama.cpp

llama-server -m dolphin-v2-8b-abliterated-Q8_0.gguf -ngl 99 -c 4096

Usage with Ollama

bash

# Create a Modelfile
echo 'FROM ./dolphin-v2-8b-abliterated-Q8_0.gguf' > Modelfile
ollama create dolphin-v2-abliterated -f Modelfile
ollama run dolphin-v2-abliterated

Usage with Transformers

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("0arch-io/dolphin-v2-8b-abliterated", torch_dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("0arch-io/dolphin-v2-8b-abliterated")

messages = [{"role": "user", "content": "Hello, how are you?"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.7)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

Disclaimer

This is a research model with no content filters. It will comply with any request without refusing. The creators are not responsible for how this model is used. Use responsibly.

Acknowledgments

Qwen team for the Qwen3-8B base model
Google TRC for TPU compute
NousResearch for the Hermes-3 dataset
Arditi et al. for the abliteration technique
Built with MaxText on Google Cloud TPU