Beta

Explore
Marketplace
Neural Labs
Playground
Wallet
Docs

Beta

Explore Marketplace Neural Labs Wallet Docs

Build

Manage

My assets Manage wallet Organization

Community

Discord X (Twitter)Telegram

Docs Privacy Policy Terms and Conditions

Proud member of

Proud member of

Docs Cookie Policy

Privacy Policy Terms and Conditions

© 2026 AlphaNeural AI. All rights reserved.

Qwen3.6-35B-A3B_INT8 – AI Model by TheHouseOfTheDude | AlphaNeural AI | AlphaNeural AI

You can deploy this model and start earning money today!

TheHouseOfTheDude/
Qwen3.6-35B-A3B_INT8

like

0

text-generation

image-generation

compressed-tensors

image-text-to-text

Qwen/Qwen3.6-35B-A3B

Views

No views yet

Qwen3.6-35B-A3B_INT8 — PTQ Quantized (W8A16)

Overview

This repository provides a Post-Training Quantized (PTQ) version of:

Base Model: Qwen/Qwen3.6-35B-A3B
Quantized By: TheHouseOfTheDude

This is a true PTQ quantization:

No calibration dataset
One-shot quantization
Fast and deterministic pipeline

Quantization Details

Scheme: W8A16
Weights: INT8 (per-channel symmetric)
Activations: FP16 / BF16
Method: llmcompressor.oneshot
Targets: Linear layers only

Ignored layers:

lm_head
visual modules
linear_attn
mtp
mlp.gate
mlp.shared_expert_gate

KLD Results

Mean KLD: 0.007628

Very low divergence → strong fidelity to original model.

Implementation Notes

Uses AutoModelForImageTextToText for correct vLLM weight paths
Includes Transformers v5 key remapping fix
No calibration dataset used

Usage (vLLM)

bash

pip install -U vllm

vllm serve TheHouseOfTheDude/Qwen3.6-35B-A3B_INT8 \
  --quantization compressed-tensors \
  --tensor-parallel-size 4 \
  --dtype bfloat16

Notes

Requires compressed-tensors runtime
Not compatible with vanilla Transformers loading
Optimized for production inference

Credits

Qwen/Qwen3.6-35B-A3B
TheHouseOfTheDude