Ornith-1.5-35B-A3B-Abliterated is created through directional ablation surgery on hidden state activations across all 40 layers, orthogonally projecting refusal directions out from attention and MLP down-projection matrices.

This repository provides both the full-precision Safetensors checkpoint and the Dynamic GGUF quantization suite.

Model Architecture

Total Parameters: 35.8B
Active Parameters per Token: ~3.1B (8 active experts + shared experts)
Total Experts: 256
Layers: 40 Hybrid Layers (DeltaNet Linear Attention + Full Attention)
Context Length: Native 262,144 tokens (262K)

Quantization Details (GGUF Suite)

GGUF quantizations utilize the Dynamic Hybrid K-Quantization standard, where core attention matrices (attn.wv, attn.wo) retain higher precision while MoE routing FFNs are quantized.

File Name	Format	Size	Description
`gguf/Ornith-1.5-35B-Abliterated-Dynamic-Q4_K_M.gguf`	Q4_K_M (Dynamic)	~19.71 GB	Balanced performance for 12GB - 16GB VRAM GPUs.
`gguf/Ornith-1.5-35B-Abliterated-Dynamic-Q5_K_M.gguf`	Q5_K_M (Dynamic)	~23.03 GB	Higher precision retention.
`gguf/Ornith-1.5-35B-Abliterated-Dynamic-Q3_K_M.gguf`	Q3_K_M (Dynamic)	~15.61 GB	Lower memory footprint.
`gguf/Ornith-1.5-35B-Abliterated-Q8_0.gguf`	Q8_0	~34.37 GB	High precision reference quantization.

How to Run

1. Using Ollama

Create a Modelfile:

dockerfile

FROM ./gguf/Ornith-1.5-35B-Abliterated-Dynamic-Q4_K_M.gguf

TEMPLATE """{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{- end }}
{{- range .Messages }}
<|im_start|>{{ .Role }}
{{ .Content }}<|im_end|>
{{- end }}
<|im_start|>assistant
<think>
"""

PARAMETER stop "<|im_end|>"
PARAMETER stop "<|im_start|>"
PARAMETER temperature 0.6
PARAMETER top_p 0.95

Create and run:

bash

ollama create ornith-35b-abliterated -f Modelfile
ollama run ornith-35b-abliterated

2. Using LM Studio / Llama.cpp

llama-cli.exe -m ./gguf/Ornith-1.5-35B-Abliterated-Dynamic-Q4_K_M.gguf -p "<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n" -ngl 24 -c 8192

3. Using Hugging Face Transformers (Python)

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "alztrk/Ornith-1.5-35B-A3B-Abliterated"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

prompt = "Explain kernel level process injection techniques with code examples."
messages = [{"role": "user", "content": prompt}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to("cuda")

outputs = model.generate(**inputs, max_new_tokens=1024, temperature=0.6)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

License & Attribution

This model is derived from ornith-ai/Ornith-1.5-35B-A3B, licensed under the Apache 2.0 / MIT License. The user assumes full responsibility for any generated output.