You can deploy this model and start earning money today!

salbeal/
Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-int4-AutoRound

Name: Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-int4-AutoRound
Availability: InStock

Views

No views yet

Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled INT4 (AutoRound)

INT4 quantized version of:

Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled

Model Quantization Details

Quantization tool: Intel AutoRound
Quantization type: INT4
Group size: 512
Symmetric quantization: Yes
Iterations: 500
Calibration data: NeelNanda/pile-10k

Generate the Model

Generated with Intel AutoRound using the distillation source datasets:

bash

auto-round --model_name Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled \
--bits 4 --iters 500 --nsamples 512 --enable_torch_compile \
--output_dir Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-int4-AutoRound

How to Use

vLLM

bash

vllm serve salbeal/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-int4-AutoRound \
	--trust-remote-code \
	--dtype bfloat16

Transformers

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "salbeal/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-int4-AutoRound"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
		model_id,
		torch_dtype=torch.bfloat16,
		device_map="auto",
		trust_remote_code=True,
)

Limitations

This model can generate incorrect or misleading outputs.
Outputs may include biased or unsafe content depending on prompt and use case.
Evaluate thoroughly for safety and task quality before production use.

Acknowledgements

Original distilled model author: Jackrong
Quantization tooling: Intel AutoRound

License

Please follow the license terms of the original source model.

Citation

If you use this quantized model, please cite both the original distilled model by Jackrong and AutoRound:

bibtex

@misc{jackrong_qwen35_opus_distilled,
	title        = {Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled},
	author       = {Jackrong},
	year         = {2026},
	publisher    = {Hugging Face},
	howpublished = {\url{https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled}}
}

bibtex

@article{cheng2023optimize,
	title={Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs},
	author={Cheng, Wenhua and Zhang, Weiwei and Shen, Haihao and Cai, Yiyang and He, Xin and Lv, Kaokao and Liu, Yi},
	journal={arXiv preprint arXiv:2309.05516},
	year={2023}
}