You can deploy this model and start earning money today!

lyf/
Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4

Name: Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4
Availability: InStock

Views

No views yet

Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4

NVFP4 vLLM-first build of HauhauCS/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive, with MTP tensors from the official Qwen/Qwen3.6-27B checkpoint and the full visual tower preserved in floating point.

Lineage

Component	Source
Text backbone	`HauhauCS/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive`
MTP weights	`Qwen/Qwen3.6-27B`
Visual tower	Qwen3.6-compatible visual tower preserved from the full conversion path
Quantization pipeline	`li-yifei/gguf-to-nvfp4`

NVFP4 text backbone shards for vLLM compressed-tensors loading.
model-multimodal-extra.safetensors containing:
- 333 model.visual.* tensors for the visual tower.
- 15 mtp.* tensors for speculative decoding experiments.
Runtime-compatible model.safetensors.index.json with no stray model.language_model.visual.* entries.

Recommended vLLM Startup

Full multimodal profile:

bash

VLLM_NVFP4_GEMM_BACKEND=marlin \
vllm serve /path/to/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4 \
  --host 0.0.0.0 \
  --trust-remote-code \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.92 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 4096 \
  --kv-cache-dtype fp8

Text-only long-context profile:

bash

VLLM_NVFP4_GEMM_BACKEND=marlin \
vllm serve /path/to/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4 \
  --host 0.0.0.0 \
  --trust-remote-code \
  --language-model-only \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.92 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 4096 \
  --kv-cache-dtype fp8

MTP speculative profile:

bash

VLLM_NVFP4_GEMM_BACKEND=marlin \
vllm serve /path/to/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4 \
  --trust-remote-code \
  --language-model-only \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.92 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 4096 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":2}'

Agent-Safe Responses Template

For agent loops that use /v1/responses, the stock Qwen3.6 template may enter thinking mode by default and spend the whole output budget on reasoning. This repo includes an optional deployment template:

chat_template_responses_reasoning_switch.jinja

Recommended agent-safe startup:

VLLM_NVFP4_GEMM_BACKEND=marlin vllm serve /path/to/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4   --host 0.0.0.0   --trust-remote-code   --language-model-only   --chat-template chat_template_responses_reasoning_switch.jinja   --max-model-len 131072   --gpu-memory-utilization 0.92   --max-num-seqs 1   --max-num-batched-tokens 4096   --kv-cache-dtype fp8

Behavior with this template:

/v1/responses without reasoning: non-thinking by default; avoids hidden-reasoning leakage in agent loops.
/v1/responses with reasoning: {"effort": "low|medium|high"}: opens Qwen thinking mode via the template variable reasoning_effort.
This does not patch vLLM. Requested thinking is emitted as normal output text unless you run a separate parser/wrapper path.

Validation Status

Tested on RTX 5090 with vllm/vllm-openai:cu130-nightly.
VLLM_NVFP4_GEMM_BACKEND=marlin is validated for this fixed artifact across text-only, full multimodal, and MTP startup paths.
flashinfer-cutlass also works and may be preferable for throughput tuning on this RTX 5090.
Full multimodal startup succeeds with Qwen3_5ForConditionalGeneration.
Visual loading no longer emits Parameter visual.blocks... not found in params_dict.
/v1/responses image input was tested successfully against a 1980x1160 PNG screenshot.
MTP tensors are included and startup has been validated with both marlin and flashinfer-cutlass; acceptance and throughput depend on workload and vLLM tuning.

Notes

The visual tower is intentionally not NVFP4-quantized; it is preserved separately in the multimodal extra shard.
Qwen3.6 linear-attention layers such as linear_attn.in_proj_a / linear_attn.in_proj_b remain BF16, so their non-64-aligned output dimensions do not enter the NVFP4 Marlin GEMM path.
For agentic long-context text serving on a single 5090, use --language-model-only to avoid reserving vision-side memory.
Current vLLM Responses behavior may still expose Qwen thinking text depending on chat-template handling; applications should parse and post-process accordingly.

References

Source finetune GGUF: https://huggingface.co/HauhauCS/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive
Official base model: https://huggingface.co/Qwen/Qwen3.6-27B
Conversion and quantization pipeline: https://github.com/li-yifei/gguf-to-nvfp4
vLLM deployment config for RTX 5090: https://github.com/li-yifei/vllm-qwen3.6-nvfp4-5090

lyf/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4

Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4

Lineage

Contents

Recommended vLLM Startup

Agent-Safe Responses Template

Validation Status

Notes

References

lyf/
Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4