You can deploy this model and start earning money today!

AxiangQAQ/
Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK

Name: Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK
Availability: InStock

Views

No views yet

Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK

FP8 Block Quantization + MTP Speculative Decoding

Uncensored | 35B MoE (3B active) | Vision + Text | ~38GB

Base Model · FP8 Reference · MTP Technique

Overview

This is an FP8_BLOCK quantized version of llmfan46/Ornith-1.0-35B-uncensored-heretic with an MTP head grafted from Qwen3.6-35B-A3B for speculative decoding.

The uncensored heretic version achieves 90% fewer refusals (9/100 vs 89/100 original) with only 0.0019 KL divergence.

Key Features

Quantization	FP8_BLOCK (128×128 block, data-free PTQ)
Model Size	~38GB (vs ~70GB BF16)
MTP Head	BF16, from Qwen3.6-35B-A3B (19 tensors)
Throughput	+21% with speculative decoding (ref. benchmark, not tested)
Architecture	Qwen3.5 MoE, 256 experts, 8 active per token
Multimodal	Vision + Text

Quick Start

vLLM (Recommended)

bash

pip install vllm

vllm serve AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK \
  --served-model-name ornith-35b-uncensored-fp8 \
  --quantization compressed-tensors \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.93 \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

Performance-optimized configuration (validated on RTX 4090 48GB):

bash

vllm serve AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK \
  --served-model-name ornith-35b-uncensored-fp8 \
  --host 0.0.0.0 \
  --port 8000 \
  --tensor-parallel-size 1 \
  --max-model-len 204800 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.93 \
  --dtype bfloat16 \
  --quantization compressed-tensors \
  --kv-cache-dtype fp8_e4m3 \
  --mm-encoder-tp-mode data \
  --mm-processor-cache-type shm \
  --mm-shm-cache-max-object-size-mb 512 \
  --media-io-kwargs '{"video": {"num_frames": -1}}' \
  --attention-backend flashinfer \
  --enable-prefix-caching \
  --enable-chunked-prefill \
  --generation-config vllm \
  --max-cudagraph-capture-size 16 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --compilation_config.mode VLLM_COMPILE \
  --compilation_config.cudagraph_mode PIECEWISE \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_xml \
  --enable-auto-tool-choice \
  --trust-remote-code

GPU Requirements: ~40GB VRAM minimum (model is ~38GB FP8). The configuration above was validated on a modified RTX 4090 48GB. For lower VRAM GPUs, quantization to GGUF format is recommended instead.

SGLang

bash

pip install sglang

python3 -m sglang.launch_server \
  --model-path AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK \
  --host 0.0.0.0 \
  --port 30000

Transformers

python

from transformers import AutoModelForMultimodalLM, AutoProcessor

model = AutoModelForMultimodalLM.from_pretrained(
    "AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK",
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(
    "AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK",
    trust_remote_code=True,
)

Performance

Note: This model has not been benchmarked. The data below is from the reference model shisa-ai/Ornith-1.0-35B-FP8-BLOCK-MTP and is provided for reference only.

Reference benchmarks (RTX PRO 6000 Blackwell, single GPU):

Config	Output tok/s	vs Baseline	TTFT (ms)	TPOT (ms)	Acceptance
No speculative	203.1	—	51.7	4.65	—
MTP tokens=1	221.0	+8.8%	60.3	4.21	85.8%
MTP tokens=3	246.5	+21.4%	65.5	3.57	67.0%

Source: shisa-ai/Ornith-1.0-35B-FP8-BLOCK-MTP

Quantization Details

Tool: llmcompressor v0.12.0
Method: Model-free PTQ (no calibration data needed)
Scheme: FP8_BLOCK (compressed-tensors format)
Weight: Static FP8, symmetric, 128×128 block strategy
Activation: Dynamic FP8, symmetric, group size 128
MTP Head: BF16 (not quantized)

Ignored layers: lm_head, embed_tokens, visual.*, mlp.gate, mlp.shared_expert_gate, linear_attn.*, mtp.*

Model Lineage

deepreinforce-ai/Ornith-1.0-35B (MIT, original)
    └── llmfan46/Ornith-1.0-35B-uncensored-heretic (Heretic v1.2.0 MPOA ablation)
        └── AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK (this model)
            ├── FP8_BLOCK quantization (llmcompressor)
            └── MTP head grafted from Qwen/Qwen3.6-35B-A3B (donor only)

Credits

Component	Source
Original Model	deepreinforce-ai/Ornith-1.0-35B
Uncensored Version	llmfan46/Ornith-1.0-35B-uncensored-heretic
FP8 Quantization Reference	shisa-ai/Ornith-1.0-35B-FP8-BLOCK
MTP Graft Technique	protoLabsAI/Ornith-1.0-9B-MTP
MTP Donor	Qwen/Qwen3.6-35B-A3B
Chat Template Fix	froggeric/Qwen-Fixed-Chat-Templates

License

MIT — same as the source Ornith-1.0-35B model.

Citation

bibtex

@misc{ornith-35b,
    title = {{Ornith-1.0-35B}: Agentic Coding, Open to All},
    url = {https://deep-reinforce.com/ornith_1_0.html},
    author = {{DeepReinforce Team}},
    year = {2026}
}