Qwen3.5-27B-Writer-exl3-5bpw-hb8 is an EXL3-quantized checkpoint for ExLlamaV3-compatible runtimes, published by groxaxo. It is intended for open-source evaluation, reproducible experimentation, and compatible local or hosted inference workflows. The wording below is deliberately limited to what can be verified from this repository's metadata and artifacts.

At a glance

Field	Details
Format	EXL3
Source / base	`ConicCat/Qwen3.5-27B-Writer`
Intended task	`text-generation`
License	`apache-2.0`

What is included

*.safetensors (3 files)
config.json
generation_config.json
tokenizer.json
tokenizer_config.json
chat_template.jinja
quantization_config.json
Additional configuration, tokenizer, processor, or shard files (10 visible artifacts total)

Quick start

EXL3-compatible runtimes

Download the EXL3 files and load the desired bitrate with a current ExLlamaV3-compatible runtime. The correct loader and context settings depend on the model architecture and should be verified against the runtime's documentation.

Compatibility and responsible use

Use a runtime that explicitly supports this format, architecture, and modality.
Keep configuration, tokenizer, processor, projection, and weight files from the same revision together.
Review the source model card and license before redistribution or deployment.
Hardware needs depend on parameter count, context length, cache precision, quantization, and concurrency.
Report reproducible issues with the runtime version, hardware, launch command, and a minimal example.

Quantization or conversion changes numerical behavior, memory use, and throughput relative to the source checkpoint; validate quality on your own workload.

Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws.

A writing & roleplay finetune of Qwen3.5 27B. The primary emphasis is on writing quality as it strongly generalizes across both domains. This model is also trained from ConicCat/Qwen3.5-Antirep-27B to mitigate repetition issues.

The basic idea is to use a curriculum learning setup to overcome the lack of high quality roleplay data by first training on lower quality roleplay data, then training on higher quality writing data. Starting from ConicCat/Qwen3.5-Antirep-27B, the model was trained on a roughly equal mixture of instruct / roleplay / writing data for three epochs. The model was then trained for eleven epochs on a smaller dataset of short story anthologies by critically acclaimed authors.

Recommended Settings

Chatml template with <think>\n\n</think> or {{char}}: prefill. Only non-thinking was trained, but thinking probably still works.
temperature = 0.7
top_p = 0.95
I do not recommend using high rep pen values like Qwen suggests for the base model. rep_pen = 1.05 or a moderate dry setting should suffice.
For quants, Q4_K_M runs well with ~100k context on 24GB Vram
IQ4_XS should fit on 16GB Vram with about 20-24k context with the vulkan backend, although it's pretty tight and may require some fiddling around with open programs e.t.c.

Datasets

ConicCat/AntiRep to mitigate repetitition.
internlm/Condor-SFT-20K for instruct; even though instruct capabilities are not the primary focus, adding some instruct data helps mitigate forgetting and maintains general intellect and instruction following capabilites.
PJMixers-Dev/C2-Logs-Sonnet-4.5-all for roleplay. Pretty much exactly what it says on the tin, the venerable C2 logs with the last turn regenerated by Sonnet 4.5 and refusals removed.
ConicCat/Gutenberg-SFT. A reformatted version of the original Gutenberg DPO dataset by jondurbin for SFT with some slight augmentation to address many of the samples being overly long.
A dataset of short story anthologies. Unfortunately, I am unable to release this set as all of the data is under copyright.

groxaxo/Qwen3.5-27B-Writer-exl3-5bpw-hb8

Quantized using the default exllamav3 (0.0.25) quantization process.

ConicCat/Qwen3.5-27B-Writer

Overview