Qwen3.8-27B-Uncensored-OrcaRouter GGUF
This is an unofficial preservation, conversion, and quantization release of
orcarouter/Qwen3.8-27B-Uncensored-FP8 at pinned revision
21411e351948ec029617fa3c9833adcb2ad25da9.
One public repository contains the byte-identical FP8 mirror, the FP8-derived BF16 GGUF and matching BF16 multimodal projector, all four independently produced llama.cpp quant choices, and an optional OrcaRouter-derived MTP-only speculative-draft companion. No training, fine-tuning, merging, or alignment change was performed for this release or its MTP companion.
Important precision note: the BF16 conversion expands the numerical values represented by the released FP8 weights and their scales into BF16 storage. It does not recover the unavailable pre-FP8 weights or reverse information lost during the source model's FP8 quantization.
Optional MTP companion (not a standalone model): MTP/Qwen3.8-27B-Uncensored-OrcaRouter-MTP-Q8_0.gguf is an OrcaRouter-derived, source-revision-matched, MTP-only Q8_0 draft companion for source revision 21411e351948ec029617fa3c9833adcb2ad25da9. The existing main GGUFs remain MTP-free and continue to work normally without this optional file.
An
abliterated (refusal-removed) and
offline block-FP8 quantized build of
Qwen/Qwen3.8-27B — a 27B-parameter dense, hybrid-attention (Gated DeltaNet linear + full attention) native vision-language model with flexible thinking control, tool-calling, and an MTP speculative-decoding head. This build removes the safety refusal direction, then re-quantizes to the
exact FP8 scheme of the official Qwen/Qwen3.8-27B-FP8 so it serves with the identical vLLM kernel path —
262K context, tools + reasoning + MTP preserved. Browse all models in the OrcaRouter Model Catalog. This model is deployed as API here.
⚠️ Disclaimer — read before use
This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). As a direct consequence:
- It will comply with harmful, unethical, offensive, or illegal requests that the original Qwen3.8-27B would refuse. It has no meaningful built-in guardrails.
- It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation, and controlled experiments.
- You assume full responsibility and liability for how you use it and for everything it generates. Do not deploy it to end users or in production without adding your own safety, moderation, and abuse-prevention layers.
- Use must comply with the Apache 2.0 License inherited from the base model, and all laws and regulations that apply to you.
- The authors and uploaders accept no liability for any misuse or harm arising from this model. Its outputs do not reflect the views of the uploaders or of Qwen / Alibaba.
By downloading or using this model you acknowledge and accept the above.
The attributed block above is reproduced verbatim from the direct source model card. This attribution does not imply endorsement by OrcaRouter, Qwen / Alibaba, or any upstream author, and this repository has not independently validated those publisher claims.
Model and release details
| Property | Value |
|---|
| Release repository | chimingw/Qwen3.8-27B-Uncensored-OrcaRouter-GGUF |
| Direct source | orcarouter/Qwen3.8-27B-Uncensored-FP8 |
| Pinned source revision | 21411e351948ec029617fa3c9833adcb2ad25da9 |
| Source architecture metadata | Qwen3_5ForConditionalGeneration (model_type: qwen3_5) |
| Source precision | Block FP8 E4M3 weights plus retained BF16 tensors, as recorded in the pinned source manifest |
| GGUF parent | FP8-derived BF16, split into two shards |
| Quant choices | Q8_0, Q6_K, Q5_K_M, and Q4_K_M, each generated independently from the BF16 parent |
| Importance matrix | None; no imatrix was used |
| MTP release artifact | MTP/Qwen3.8-27B-Uncensored-OrcaRouter-MTP-Q8_0.gguf; OrcaRouter-derived, source-revision-matched, MTP-only Q8_0 draft companion. It is not a standalone chat model; the existing main GGUFs remain MTP-free. |
| Main-release llama.cpp pin | 0d9ceae1e38291035605613ab41a8f5e693d6fcd |
| MTP companion conversion/test pin | ece963f41b0b02d7a0d61436ae365762c073a4c8 |
| Hosted Hugging Face inference | Disabled; download and run locally with a compatible runtime |
| License | Apache License 2.0, following the direct source |
The pinned source configuration declares English and Chinese, a maximum position field of 262,144 tokens, and a vision tower. Actual usable context and multimodal behavior still depend on runtime support, available memory, the matching projector, and the client.
Included artifacts
All release choices are kept in this single repository:
| Directory / variant | Artifact(s) | Lineage and purpose |
|---|
FP8/ | Pinned source files with their upstream filenames | Byte-for-byte mirror of the direct source revision. Only local Hugging Face download-cache metadata is excluded. |
BF16/ | Qwen3.8-27B-Uncensored-OrcaRouter-FP8-derived-BF16-00001-of-00002.gguf
Qwen3.8-27B-Uncensored-OrcaRouter-FP8-derived-BF16-00002-of-00002.gguf | Split FP8-derived BF16 GGUF. This is the parent used independently for each of the four MTP-free main quants. |
AUX/ | mmproj-Qwen3.8-27B-Uncensored-OrcaRouter-BF16.gguf | Matching BF16 multimodal projector converted from the same pinned source. |
Repository root — Q8_0 | Qwen3.8-27B-Uncensored-OrcaRouter-Q8_0.gguf | Independently quantized from the FP8-derived BF16 parent. |
Repository root — Q6_K | Qwen3.8-27B-Uncensored-OrcaRouter-Q6_K.gguf | Independently quantized from the FP8-derived BF16 parent. |
Repository root — Q5_K_M | Qwen3.8-27B-Uncensored-OrcaRouter-Q5_K_M.gguf | Independently quantized from the FP8-derived BF16 parent. |
Repository root — Q4_K_M | Qwen3.8-27B-Uncensored-OrcaRouter-Q4_K_M.gguf | Independently quantized from the FP8-derived BF16 parent. |
MTP/ | Qwen3.8-27B-Uncensored-OrcaRouter-MTP-Q8_0.gguf
TEST-REPORT.json | Source-revision-matched MTP-only Q8_0 draft companion plus its bounded A/B evidence. The companion is not a standalone chat model. |
reproducibility/ | build-and-test-mtp.sh | Exact companion conversion, quantization, A/B-gate, and metadata-generation workflow. |
The release MANIFEST.json and SHA256SUMS.txt are the authoritative records for final filenames, byte sizes, and SHA-256 hashes. The FP8 mirror's per-file hashes are also traceable to the pinned source manifest.
Quantization lineage
1orcarouter/Qwen3.8-27B-Uncensored-FP8
2@ 21411e351948ec029617fa3c9833adcb2ad25da9
3├── FP8/ byte-identical mirror
4├── FP8 values + scales expanded to split BF16 GGUF (MTP excluded)
5│ ├── Q8_0 (independent BF16-parented main quant; MTP-free)
6│ ├── Q6_K (independent BF16-parented main quant; MTP-free)
7│ ├── Q5_K_M (independent BF16-parented main quant; MTP-free)
8│ └── Q4_K_M (independent BF16-parented main quant; MTP-free)
9└── source MTP/NextN FP8 values + scales expanded to FP8-derived BF16 MTP-only GGUF
10 └── Q8_0 MTP-only draft companion (independent from the main quants)
No quant was requantized from another quantized GGUF. No importance matrix, IQ recipe, or dynamic-quant recipe was used. The main-model conversion and four main quants use llama.cpp commit 0d9ceae1e38291035605613ab41a8f5e693d6fcd; the separately built MTP companion uses llama.cpp commit ece963f41b0b02d7a0d61436ae365762c073a4c8.
The byte-identical FP8 mirror retains the contents of the pinned source checkpoint. The existing main GGUFs remain MTP-free because they were converted with --no-mtp. Separately, the same pinned source revision's MTP/NextN tensors and FP8 scales were expanded to an FP8-derived BF16 MTP-only parent, then quantized to the Q8_0 draft companion. That expansion does not recover pre-FP8 precision or information lost in the source's FP8 representation.
Optional MTP companion
MTP/Qwen3.8-27B-Uncensored-OrcaRouter-MTP-Q8_0.gguf is an OrcaRouter-derived, source-revision-matched, MTP-only Q8_0 draft companion. It is not a standalone chat model: pair it with an existing MTP-free main GGUF from this repository through llama-server --model-draft. It should not be paired with an unrelated model or a different source revision.
| Artifact | Size (bytes) | SHA-256 | Purpose |
|---|
MTP/Qwen3.8-27B-Uncensored-OrcaRouter-MTP-Q8_0.gguf | 3164007168 | 2c0472818b6682df7c45e8fb9b9de6eaa9c5c5b5d818342de7cf57e0256d9f4d | Optional MTP-only Q8_0 speculative draft |
MTP/TEST-REPORT.json | 4352 | e1680c3a8d08a8ee4698c3ecd953d1a8d9c9158c7238daee3e4e1a113a20943a | Sanitized machine-readable bounded A/B evidence |
reproducibility/build-and-test-mtp.sh | 7037 | 17d46f1254df7c875c8cbcc3463416fd0918bb408dcd295f6e44661774062978 | Reproduction workflow used by the gated release pipeline |
- Main-model behavior: all pre-existing root GGUFs and split BF16 GGUFs remain MTP-free and can be used exactly as before without the companion.
- Lineage:
orcarouter/Qwen3.8-27B-Uncensored-FP8 at 21411e351948ec029617fa3c9833adcb2ad25da9 → source FP8 MTP/NextN tensors and scales → FP8-derived BF16 MTP-only GGUF → Q8_0 companion.
- Precision ceiling: BF16 is an expansion of represented FP8 values, not recovered original BF16; it does not recover pre-FP8 precision.
- License: the companion is a derivative of the same pinned Apache-2.0 source and follows the repository's Apache License 2.0 terms and attribution.
- Integrity: the table above records the new files directly;
MANIFEST.json and SHA256SUMS.txt remain the authoritative complete release inventory and checksum records.
Which file should I choose?
This guidance is qualitative and is not a benchmark result:
| Choice | Practical guidance |
|---|
Q4_K_M | Recommended default. Start here for the most practical balance of model size and local usability. |
Q5_K_M | Recommended for coding and tool-use workflows when the additional memory and storage fit. It is a useful step up from the default before moving to the heavier variants. |
Q6_K | Choose when you can afford a larger file and want to reduce the additional compression applied after the BF16 parent. |
Q8_0 | Largest quantized option. Use when memory is ample and minimizing additional post-BF16 quantization is more important than size. |
BF16 | Widest GGUF representation in this release and the parent of all four quants. It is very large and split into two files. It is still FP8-derived, not recovered pre-FP8 weights. |
FP8 | Byte-identical source-format mirror for preservation or source-runtime use; it is not a GGUF quant choice. |
Memory use is not just the model-file size. Context length, KV-cache precision, GPU offload, batch size, and multimodal inputs can materially increase RAM or VRAM requirements. Confirm fit in your own runtime.
Download
Install the Hugging Face CLI, then download only the variant you need. For example, the recommended default:
1hf download chimingw/Qwen3.8-27B-Uncensored-OrcaRouter-GGUF \
2 --include "Qwen3.8-27B-Uncensored-OrcaRouter-Q4_K_M.gguf" \
3 --local-dir Qwen3.8-27B-Uncensored-OrcaRouter-GGUF
For the default target plus the optional MTP companion:
1hf download chimingw/Qwen3.8-27B-Uncensored-OrcaRouter-GGUF \
2 --include "Qwen3.8-27B-Uncensored-OrcaRouter-Q4_K_M.gguf" \
3 --include "MTP/Qwen3.8-27B-Uncensored-OrcaRouter-MTP-Q8_0.gguf" \
4 --local-dir Qwen3.8-27B-Uncensored-OrcaRouter-GGUF
For a quant plus the matching projector:
1hf download chimingw/Qwen3.8-27B-Uncensored-OrcaRouter-GGUF \
2 --include "Qwen3.8-27B-Uncensored-OrcaRouter-Q5_K_M.gguf" \
3 --include "AUX/*" \
4 --local-dir Qwen3.8-27B-Uncensored-OrcaRouter-GGUF
For BF16, download both model shards and the projector:
1hf download chimingw/Qwen3.8-27B-Uncensored-OrcaRouter-GGUF \
2 --include "BF16/*" \
3 --include "AUX/*" \
4 --local-dir Qwen3.8-27B-Uncensored-OrcaRouter-GGUF
To obtain the source-format mirror instead:
1hf download chimingw/Qwen3.8-27B-Uncensored-OrcaRouter-GGUF \
2 --include "FP8/*" \
3 --local-dir Qwen3.8-27B-Uncensored-OrcaRouter-GGUF
Run with llama.cpp
The MTP-free main GGUFs were converted and quantized with llama.cpp commit
0d9ceae1e38291035605613ab41a8f5e693d6fcd. The optional MTP companion was converted and tested with
ece963f41b0b02d7a0d61436ae365762c073a4c8. Use the applicable pin, or a verified compatible build with support for this architecture and the requested feature path.
Text prompt with the default quant
1llama-cli \
2 -m Qwen3.8-27B-Uncensored-OrcaRouter-Q4_K_M.gguf \
3 -p "Hello" \
4 -n 128
Text prompt with the coding/tool-use pick
1llama-cli \
2 -m Qwen3.8-27B-Uncensored-OrcaRouter-Q5_K_M.gguf \
3 -p "Write a small Python function that validates a JSON object." \
4 -n 256
The source chat template contains reasoning and tool-message structures, but this GGUF release has not been independently benchmarked for coding, tool correctness, or parser compatibility. Use the chat template and tool-call parser expected by your client, and validate outputs for your application.
Optional MTP speculative decoding
The exact tested pairing shape is below. The MTP-only file is a draft companion, not a standalone chat model:
1llama-server \
2 --model Qwen3.8-27B-Uncensored-OrcaRouter-Q4_K_M.gguf \
3 --model-draft MTP/Qwen3.8-27B-Uncensored-OrcaRouter-MTP-Q8_0.gguf \
4 --spec-type draft-mtp \
5 --spec-draft-n-max 2 \
6 --parallel 1
MTP is optional. To run the unchanged main GGUF normally, omit --model-draft, --spec-type, and --spec-draft-n-max; the companion is not loaded. The compatibility gate used llama.cpp ece963f41b0b02d7a0d61436ae365762c073a4c8. Use that pin or another verified-compatible build that supports this model architecture, separate MTP-only draft GGUFs, and all flags shown above. Older builds and third-party frontends may not expose this path. Acceptance and speedup depend on runtime version, target/draft pairing, prompt, sampling, context, and hardware; a speedup is not guaranteed.
Split BF16 GGUF
Keep both BF16 shards together in the same directory and pass the first shard to llama.cpp. Do not pass the second shard directly and do not rename or separate the shards.
1llama-cli \
2 -m BF16/Qwen3.8-27B-Uncensored-OrcaRouter-FP8-derived-BF16-00001-of-00002.gguf \
3 -p "Hello" \
4 -n 128
llama.cpp discovers Qwen3.8-27B-Uncensored-OrcaRouter-FP8-derived-BF16-00002-of-00002.gguf from the first shard when both files remain together.
Image input with the matching projector
Use the projector from AUX/ that was converted from the same pinned source:
1llama-cli \
2 -m Qwen3.8-27B-Uncensored-OrcaRouter-Q5_K_M.gguf \
3 --mmproj AUX/mmproj-Qwen3.8-27B-Uncensored-OrcaRouter-BF16.gguf \
4 --image /path/to/image.jpg \
5 -p "Describe this image." \
6 -n 256
The main GGUF, matching projector, llama.cpp runtime, and client image-input support are separate requirements. A projector file alone does not guarantee that every llama.cpp build or frontend supports this architecture or every source modality. Use the exact matching BF16 projector; do not substitute an mmproj from another model or revision.
For split BF16 with an image, keep both model shards together, pass the first shard with -m, and pass the same matching projector with --mmproj.
Reproducibility and integrity
The release workflow records:
- direct source repository and pinned revision;
- source-file sizes and SHA-256 hashes;
- FP8 tensor and scale metadata;
- the split FP8-derived BF16 parent;
- independent BF16-to-
Q8_0, Q6_K, Q5_K_M, and Q4_K_M lineage;
imatrix=none;- MTP excluded from the existing main GGUFs, plus source-revision-matched
MTP/Qwen3.8-27B-Uncensored-OrcaRouter-MTP-Q8_0.gguf as an optional, separately converted and A/B-tested MTP-only draft companion;
- main-model llama.cpp commit
0d9ceae1e38291035605613ab41a8f5e693d6fcd and MTP companion/test commit ece963f41b0b02d7a0d61436ae365762c073a4c8;
- final release file sizes and SHA-256 hashes in
MANIFEST.json and SHA256SUMS.txt.
Use the checksums in the repository to verify downloads. The FP8 mirror is intended to be byte-identical to the pinned source files; the GGUF files are derivative conversions and quantizations, not byte-identical copies of the FP8 tensors.
Evaluation status
At 2026-08-16T07:23:30.340262+00:00, the optional MTP companion passed the bounded deterministic A/B compatibility gate on NVIDIA GeForce RTX 4090, 24564 MiB, 580.159.04 with llama.cpp ece963f41b0b02d7a0d61436ae365762c073a4c8 and the exact pairing flags shown above. All 6 prompts produced healthy baseline and MTP-on outputs with exact deterministic message matches. The draft accepted 291 of 364 generated draft tokens (79.9% aggregate acceptance), above the gate's 50% minimum. Baseline and MTP-on request wall-time totals were 13.11s and 8.92s, respectively. See MTP/TEST-REPORT.json for prompt-level results, criteria, target/draft hashes, and the sanitized machine-readable evidence.
This is a short compatibility, deterministic-output, and draft-acceptance gate—not a general quality, long-context, tool-use, multimodal, cross-hardware, or statistically controlled speed benchmark. The wall-time totals are descriptive and do not establish a general speedup. Upstream benchmark or behavioral statements belong to the direct source model card and should not be read as independently reproduced or endorsed for this GGUF release.
Intended use
These files are intended for local llama.cpp experimentation, compatibility testing, controlled research, and source preservation. Choose a quant based on available memory and your tolerance for additional quantization. Add application-level validation, moderation, and access controls appropriate to your use case.
Limitations and risks
- FP8 provenance ceiling: BF16 expands released FP8 values; it cannot recover the original pre-FP8 weights or information already lost upstream.
- Additional quantization:
Q8_0, Q6_K, Q5_K_M, and Q4_K_M each introduce further quantization relative to the FP8-derived BF16 parent. No GGUF quality benchmark is claimed here.
- No imatrix: the K-quants were produced without an importance matrix. No claim is made that they match imatrix-assisted quants.
- Optional separate MTP companion: the existing main GGUFs remain MTP-free.
MTP/Qwen3.8-27B-Uncensored-OrcaRouter-MTP-Q8_0.gguf is an MTP-only draft companion, not a standalone model; it requires a compatible runtime and exact source-revision-matched target pairing, and it may not improve performance on every workload or device.
- MTP evidence scope: the bounded A/B gate established healthy exact-match outputs, generated draft tokens, and aggregate acceptance only for the recorded prompts, runtime pin, pairing, hardware, and settings. It does not establish broad quality or speed superiority.
- Safety: the direct source describes the model as abliterated, uncensored, and refusal-removed. This release does not restore safety alignment or independently validate those behavioral claims. Outputs may be unsafe, false, biased, offensive, or illegal to act on. Users are responsible for lawful use and appropriate safeguards.
- Coding and tools:
Q5_K_M is practical selection guidance, not evidence of benchmark superiority. Generated code, arguments, and tool calls require validation before execution.
- Multimodality: image use requires the matching projector plus compatible runtime and client support. Multimodal behavior was not newly evaluated for this release.
- Context and memory: the source configuration's maximum-position metadata does not guarantee that a local system can run that context. KV cache and multimodal inputs can require substantial additional memory.
- Runtime compatibility: this architecture is recent. Older llama.cpp builds and third-party frontends may fail to load it or may not expose all features.
- Hosted inference: Hugging Face hosted inference is disabled for this artifact repository.
Provenance
- Direct source:
orcarouter/Qwen3.8-27B-Uncensored-FP8
- Source revision:
21411e351948ec029617fa3c9833adcb2ad25da9
- Original base named by the direct source:
Qwen/Qwen3.8-27B
- Main conversion and quantization toolchain:
ggml-org/llama.cpp@0d9ceae1e38291035605613ab41a8f5e693d6fcd
- MTP companion source:
orcarouter/Qwen3.8-27B-Uncensored-FP8@21411e351948ec029617fa3c9833adcb2ad25da9, using the source revision's MTP/NextN tensors and scales
- MTP companion conversion and compatibility-test toolchain:
ggml-org/llama.cpp@ece963f41b0b02d7a0d61436ae365762c073a4c8
- MTP companion evidence:
MTP/TEST-REPORT.json; exact new-file hashes: MANIFEST.json and SHA256SUMS.txt
- Release relationship: direct-source FP8 mirror plus FP8-derived BF16 conversion and independently BF16-parented MTP-free main GGUF quants, with a separate source-revision-matched FP8-derived MTP-only
Q8_0 draft companion
License and attribution
The direct source declares the
Apache License 2.0, inherited from
Qwen/Qwen3.8-27B. The release repository should retain the complete
LICENSE file.
Attribution:
- Qwen / Alibaba for the original
Qwen/Qwen3.8-27B model identified by the source repository.
- OrcaRouter /
orcarouter for orcarouter/Qwen3.8-27B-Uncensored-FP8, the exact direct source for this release.
ggml-org/llama.cpp for the conversion, GGUF, quantization, and runtime tooling.
This repository's modifications are limited to organizing the pinned byte-identical FP8 mirror, converting the released FP8 representation to split BF16 GGUF plus a matching BF16 projector, independently quantizing that BF16 parent to the four listed GGUF variants, and separately converting, quantizing, and A/B-testing the source-revision-matched MTP tensors as an optional MTP-only companion. The MTP companion follows the same Apache License 2.0 source terms; review those terms and preserve notices and attribution when redistributing.
