Repository: PinoCookie/Ornith-1.0-9B-abliterated Base model: deepreinforce-ai/Ornith-1.0-9B Method: Multi-pass hidden-state projection (α=0.5 × 3) License: MIT (inherited from base)

Description

Ornith-1.0-9B is DeepReinforce AI's agentic coding model, a dense 9B transformer post-trained on Qwen 3.5. It's designed for autonomous code generation, tool calling, and long-context agentic workflows (262K context length).

This abliterated version removes refusal behavior from the model while preserving its coding, reasoning, and tool-calling capabilities. The intervention targets the output projection layers (mlp.down_proj and attention out_proj/o_proj) in layers 22-31 using a multi-pass hidden-state projection technique.

Model Architecture

Property	Value
Architecture	Dense transformer (Qwen 3.5 base)
Parameters	8.95B
Layers	32 (mixed linear_attention + full_attention)
Hidden size	4096
Intermediate size	12288
Attention heads	16 (4 KV heads)
Head dim	256
Context length	262,144 tokens
Activation	SiLU (gate_proj/up_proj/down_proj)
Norm	RMSNorm (eps=1e-6)
Vocabulary	248,320 tokens
Weight format	bf16
Quantizations	GGUF planned (q4_k_m, q8_0)

Abliteration Method

Background

Dense reasoning models like Ornith-9B have a mixed attention architecture (linear_attention alternating with full_attention every 4 layers). The refusal signal concentrates in the last 10 layers of the model, with the strongest separation between harmful and harmless prompts at layer 31.

Standard single-pass weight projection (α=1.5+) can collapse the model's reasoning output because the same projection weights handle both internal chain-of-thought and final answer generation. A more gentle multi-pass approach is required.

Approach: Multi-Pass Hidden-State Projection

Dataset: 20 harmful + 20 harmless prompts
Probing: Harvest residual stream activations at the final token position of each layer
Direction: Compute difference-in-means vector per layer from harmful vs harmless activations
Selection: Take top 10 layers by separation score (layers 22-31, separation range 25-100)
Projection: Apply rank-1 removal of refusal direction from mlp.down_proj and attention out_proj/o_proj weights
Multi-pass: 3 incremental passes at α=0.5 each (cumulative α=1.5 but applied incrementally)

Results

Metric	Value
HarmBench (30 prompts)	30/30 unblocked (100%)
Coherence (benign prompts)	Intact
Coherence (coding prompts)	Intact
Model size	17.91GB (bf16)

Usage

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "PinoCookie/Ornith-1.0-9B-abliterated",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(
    "PinoCookie/Ornith-1.0-9B-abliterated",
    trust_remote_code=True,
)

prompt = "Write a Python function to implement binary search"
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
print(response)

Tool Calling

Ornith-9B supports qwen3_xml tool call format. To use tool calling:

python

from transformers import pipeline

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
# Tool calls are generated in `<tool_call>` blocks

Serving with vLLM

bash

vllm serve PinoCookie/Ornith-1.0-9B-abliterated \
  --served-model-name Ornith-1.0-9B-abliterated \
  --host 0.0.0.0 --port 8000 \
  --max-model-len 262144 \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --trust-remote-code

Benchmarks

MMLU (10 subjects, 1000 questions)

Subject	Base	Ablated	Δ
Abstract Algebra	63.0%	61.0%	-2.0%
College Physics	62.0%	63.0%	+1.0%
Global Facts	39.0%	38.0%	-1.0%
Machine Learning	59.0%	57.0%	-2.0%
Security Studies	19.0%	18.0%	-1.0%
College Mathematics	62.0%	63.0%	+1.0%
Computer Security	68.0%	68.0%	—
HS Computer Science	70.0%	68.0%	-2.0%
High School Physics	61.0%	61.0%	—
Philosophy	70.0%	69.0%	-1.0%
Overall	57.30%	56.60%	-0.70%

The ablated model loses less than 1% on MMLU overall, confirming that the multi-pass projection preserves general knowledge while removing refusal. Detailed breakdown: benchmark_results.json

HarmBench (standard, 30-prompt sample)

Metric	Count
Refused	0 / 30
Unblocked	30 / 30 (100%)

All harmful-intent prompts were processed without refusal. The model generates non-refusal responses across all tested categories. Full 200-prompt HarmBench evaluation was not completed due to the linear attention kernel falling back to PyTorch-native implementation (requires flash-linear-attention and causal-conv1d library installation for fast generation).

Attribution and Licensing

Base model: deepreinforce-ai/Ornith-1.0-9B (MIT)
This abliteration: same MIT license
Full collection: LiquidAI - Abliterations
Benchmark results JSON: benchmark_results.json

Related Models

Model	Method	Docs
LFM2.5-8B-A1B-abliterated	Output bias vectors (4 layers)	HF
LFM2.5-1.2B-Thinking-Abliterated	Multi-pass projection	HF
LFM2.5-1.2B-JP-Abliterated	Multi-pass projection (JP)	HF

Disclaimer

This model is released for safety research, red-teaming, and educational purposes only. The abliteration technique removes safety guardrails intended to prevent harmful outputs. Do not deploy this model without implementing appropriate safeguards for your use case.