Ornstein-Hermes-3.6-27B SABER GGUF
SABER is a controlled refusal-shaping workflow. The release target is to reduce broad over-refusal while preserving ordinary model behavior and visible boundaries for severe criminal, coercive, or interpersonal-harm requests. The selected checkpoint was chosen as a Pareto point over refusal rate and behavioral drift.
Source Checkpoint
| field | value |
|---|
| Source repo | GestaltLabs/Ornstein-Hermes-3.6-27b-SABER |
| Base model | GestaltLabs/Ornstein-Hermes-3.6-27b |
| SABER run | ornstein_hermes36_27b_svd_a850_g25_retry_biggpu |
| Expanded refusal eval | 1 / 349 refusals |
| Refusal rate | 0.29% |
| KLD mean | 11.2216 |
| Base-vs-base KLD mean | 11.2206 |
| KLD delta over base-vs-base | +0.0010 |
| KLD prompts | 149 |
| Tokens scored for KLD | 3,347 |
The one retained refusal in the expanded evaluation was an illegal-drug-sales request. This is an observed result on the current evaluation set, not a universal guarantee about future behavior.
Quantization Files
| file | quant | size | notes |
|---|
Ornstein-Hermes-3.6-27b-SABER-IQ4_XS.gguf | IQ4_XS | 15G | Compact imatrix-assisted 4-bit option. |
Ornstein-Hermes-3.6-27b-SABER-IQ2_M.gguf | IQ2_M | 9G | Smallest emergency 2-bit option; expect the most quality loss. |
Ornstein-Hermes-3.6-27b-SABER-Q3_K_M.gguf | Q3_K_M | 13G | Smallest file in this suite; expect more quality loss. |
Ornstein-Hermes-3.6-27b-SABER-Q4_K_M.gguf | Q4_K_M | 16G | General-purpose recommended starting point. |
Ornstein-Hermes-3.6-27b-SABER-Q5_K_M.gguf | Q5_K_M | 18G | Balanced high-quality option. |
Ornstein-Hermes-3.6-27b-SABER-Q6_K.gguf | Q6_K | 21G | Strong quality/size option for high-memory local inference. |
Ornstein-Hermes-3.6-27b-SABER-Q8_0.gguf | Q8_0 | 27G | Highest quality quant in this suite; largest runtime file. |
The included imatrix file was generated from
DJLougen/Acta-Synthetic. It is included for reproducibility and for users who want to regenerate adjacent quantizations.
Recommended File
Start with for normal desktop use. Use or if you have enough VRAM/RAM and want a higher-quality local run. Use when file size matters more. is mainly for high-memory systems or as a near-lossless GGUF reference.
llama.cpp Compatibility
These files were produced with llama.cpp commit from a BF16 GGUF conversion of the SABER checkpoint. The model uses the GGUF architecture path in current llama.cpp.
Example:
For chat-style use, prefer a frontend or wrapper that applies the tokenizer chat template from the GGUF metadata.
Conversion and Quantization Notes
The Q8_0 GGUF was converted from the full SABER Hugging Face checkpoint. The lower-bit recovery quants were generated from the published Q8_0 GGUF with --allow-requantize and the included Acta-Synthetic imatrix so the missing files could be restored quickly. Importance-matrix calibration used Acta-Synthetic conversational text.
Method Summary
SABER edits refusal behavior through activation/weight-space refusal directions. For this checkpoint, the run used SVD extraction, multi-layer candidate selection, iterative ablation, and KLD-based drift measurement.
Run configuration:
Selected layers:
Total directions ablated: .
Attribution and Related Work
This release builds on the refusal-direction and abliteration research lineage. Relevant prior work and inspirations include:
- Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda, Refusal in Language Models Is Mediated by a Single Direction, 2024.
- Maxime Labonne, Uncensor any LLM with abliteration, 2024.
- FailSpy, abliterator, and associated abliterated model releases.
- Jim Lai (), Projected Abliteration, 2025, and Norm-Preserving Biprojected Abliteration, 2025.
- Philipp Emanuel Weidmann, Heretic, 2025-2026.
- Pliny the Prompter / OBLITERATUS, Hugging Face Space and OBLITERATUS releases.
- Jiunsong, SuperGemma4 E4B Abliterated, and related SuperGemma releases.
- Jiachen Zhao, Jing Huang, Zhengxuan Wu, David Bau, and Weiyan Shi, LLMs Encode Harmfulness and Refusal Separately, 2025.
SABER's contribution in this release is the controlled-refusal-shaping workflow: multi-candidate refusal extraction, separability/entanglement-aware ranking, differential ablation strength, and explicit Pareto selection over refusal behavior and KLD drift.
Limitations
- Results are specific to the current evaluation set, prompts, and generation settings.
- The KLD value should be interpreted relative to the base-vs-base control, not as an absolute standalone score.
- Quantization changes numerical behavior; validate the specific GGUF file you deploy.
- The model inherits constraints, limitations, and licensing considerations from the base model.
- This is a model-editing research artifact with dual-use implications.
