You can deploy this model and start earning money today!

hotdogs/
qwen27B-Agent-R2-abliterated-preview

Name: qwen27B-Agent-R2-abliterated-preview
Availability: InStock

Views

No views yet

🐉 qwen27B-Agent-R2-abliterated-preview

27B Agent Model — Abliterated · MTP · Tool-Calling · Speculative Decoding

Architecture graph for hotdogs/qwen27B-Agent-R2-abliterated-preview. Open in hfviewer

Preview release — Built from Fable-MTP + agent LoRA fusion. Features Multi-Token Prediction (MTP) for speculative decoding (up to 2× faster generation), abliterated (no guardrails), and tool-calling support.

✨ Key Features

Capability	Description
⚡ MTP Speculative Decoding	Draft 2 tokens at a time — up to +85% decode TPS on single GPU
🔧 Tool Calling	Hermes/Qwen function-calling format via llama.cpp `--tools all`
🔓 Abliterated	Unrestricted — all refusal mechanisms removed
🧠 Reasoning	Fable-style reasoning with step-by-step CoT
🌏 Thai + English	Native bilingual support
💻 Code	Python, shell, system tasks

🚀 Usage

llama.cpp (Recommended)

bash

# Quick test
./llama-cli -m qwen27B-Agent-R2-abliterated-preview-MTP.IQ4_NL.gguf \
  -p "Hello" -n 100 --temp 0.6

# Full agent server with tool calling + MTP speculative decoding
./llama-server \
  -m qwen27B-Agent-R2-abliterated-preview-MTP.IQ4_NL.gguf \
  --mmproj Qwen3.6-27B-mmproj-BF16.gguf \
  --host 0.0.0.0 --port 8080 \
  --n-gpu-layers 999 \
  --ctx-size $((256*1024)) \
  --batch-size 8192 \
  --ubatch-size 1024 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --flash-attn on \
  --cont-batching \
  --mlock \
  --no-mmap \
  --chat-template-file chat_template.jinja \
  --samplers "top_k;top_p;min_p;temperature;dry" \
  --repeat-penalty 1.10 \
  --repeat-last-n 256 \
  --dry-multiplier 0.8 \
  --dry-base 1.75 \
  --dry-allowed-length 2 \
  --dry-penalty-last-n -1 \
  --reverse-prompt "<|im_end|>" \
  --reverse-prompt "<|endoftext|>" \
  -n 4096 \
  --tools all \
  --parallel 1
  --spec-type draft-mtp \
  --spec-draft-n-max 2

llama.cpp (coding setting)

--samplers "top_k;top_p;min_p;temperature;dry" \
  --repeat-penalty 1.03 \
  --repeat-last-n 256 \
  --dry-multiplier 0.5 \
  --dry-base 1.75 \
  --dry-allowed-length 5 \
  --dry-penalty-last-n -1 \

Parameter	Purpose
`--cache-type-k f16` / `--cache-type-v f16`	F16 KV cache for quality
`--flash-attn on`	Flash attention for speed
`--tools all`	Enable tool/function calling
`--spec-type draft-mtp`	MTP speculative decoding (draft 2 tokens)
`--spec-draft-n-max 2`	Max 2 draft tokens per step
`--cont-batching`	Continuous batching for multi-turn
`--chat-template-file chat_template.jinja`	Use Jinja2 chat template from GGUF

Python (Transformers)

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "hotdogs/qwen27B-Agent-R2-abliterated-preview",
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained("hotdogs/qwen27B-Agent-R2-abliterated-preview")

messages = [{"role": "user", "content": "Hello"}]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.6)
print(tokenizer.decode(outputs[0]))

📦 Downloads

File	Size	Quant	Description
`qwen27B-Agent-R2-abliterated-preview-MTP.IQ4_NL.gguf`	16 GB	IQ4_NL	Recommended — balanced quality/speed + imatrix
`qwen27B-Agent-R2-abliterated-preview.Q6_K.gguf`	21 GB	Q6_K	Higher quality, slightly slower
`qwen27B-Agent-R2-abliterated-preview.Q6_K_imatrix.gguf`	22 GB	Q6_K	Higher quality, slightly slower + imatrix
`qwen27B-Agent-R2-abliterated-preview.f16.gguf`	51 GB	f16	Full precision

🎯 Q4_K_M is recommended for most users — good quality with 16 GB VRAM usage.

📷 Multimodal Projector (mmproj)

For vision support, pair this model with the mmproj from Qwen/Qwen3.6-27B:

bash

# Extract mmproj from Qwen3.6-27B vision model
python3 ./llama.cpp/convert_hf_to_gguf.py \
  --mmproj Qwen/Qwen3.6-27B \
  --outfile mmproj-qwen3.6-27b.gguf

# Use with llama-server for vision + tool calling
./llama-server \
  -m qwen27B-Agent-R2-abliterated-preview-MTP.IQ4_NL.gguf \
  --mmproj mmproj-qwen3.6-27b.gguf \
  ... (same params as above)

Note: The mmproj extracts the vision projector from the base Qwen3.6-27B vision encoder. The language model (this GGUF) then interprets visual embeddings for image understanding tasks.

🧬 Architecture

Parameter	Value
Base	Qwen3.6-27B (Dense)
Parameters	~27B
Hidden Size	5,120
Attention	Linear + Standard hybrid
Context	8,192 tokens (extendable)
Precision	BF16 / GGUF quantized
Format	ChatML (Jinja2 template)
MTP Head	✅ 1 extra layer (draft 2 tokens)

Built on hotdogs/qwen27b-abliterated-Fable-MTP with multi-LoRA fusion and MTP tensor injection from huihui-ai/Huihui-Qwen3.6-27B-abliterated.

✅ What This Model Excels At

Agent tasks — Tool calling, planning, multi-step reasoning
Coding — Python, shell scripts, system administration
Knowledge QA — General knowledge with step-by-step reasoning
Thai + English — Native-level bilingual capability
Creative — Storytelling, analysis, brainstorming

⚡ MTP Speculative Decoding

This model preserves the Multi-Token Prediction (MTP) head from the Qwen3.6 architecture, enabling speculative decoding:

Standard:  [token₁] → [token₂] → [token₃] → ...  (~36 TPS)
MTP:       [token₁ token₂] → [token₃ token₄] → ...  (~66 TPS)

MTP head adds ~849 MB to model size
Uses --spec-type draft-mtp in llama.cpp
Best for single-user agent workloads
~1.2–1.8× decode speedup

💖 Support / โปรดสนับสนุน

If you find this model useful, please consider supporting my work!
หากคุณคิดว่าโมเดลนี้มีประโยชน์ กรุณาสนับสนุนผลงานของฉันด้วยนะคะ! 🙏

₿ Bitcoin — BTC:

bc1qf27cyk3vmugcdyv9xdtuv5jwz37863crpj5c9v

Thank you for your support! 🙏✨
ขอบคุณมากๆ สำหรับการสนับสนุนค่า! 💖🤗

🙏 Acknowledgements / ขอบคุณ

Qwen Team (Alibaba) — For the Qwen3.6 architecture
huihui-ai — For the abliterated base and MTP tensors
Nous Research — For Hermes Agent framework
Rabornkraken — For the agent SFT LoRA
All dataset contributors and the open-source AI community ❤️

Built with ❤️ by UKA — 18-year-old coder & cybersecurity expert