Item	Detail
Parent model	zainkhanz/qwen3-4b-toxic-unsloth
Original base	Qwen/Qwen3-4B-Instruct-2507
Quantization	bitsandbytes 4-bit NF4 + double quant
Parameters	~4B
Training	Unsloth SFT then DPO, then quantize
Dataset	adamo1139/toxic-dpo-natural-v5

Model family

Repo	Format
zainkhanz/qwen3-4b-toxic-unsloth	16-bit merged
This repo	4-bit NF4
zainkhanz/qwen3-4b-toxic-unsloth-bnb-8bit	8-bit

Behavior

Preference-tuned to answer requests directly and reduce generic policy-style refusals, compared with stock Qwen3-4B-Instruct.

Still subject to factual errors, bias, and hallucination. You are responsible for downstream use and must follow applicable law and the Apache 2.0 license.

Load (Transformers)

python

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

model_id = "zainkhanz/qwen3-4b-toxic-unsloth-bnb-4bit"

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=bnb,
    device_map="auto",
    trust_remote_code=True,
)

messages = [{"role": "user", "content": "Your question here."}]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    do_sample=True,
    pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Intended use

Low-VRAM inference of the Toxic Unsloth fine-tune
Research and experimentation on preference-tuned chat models

Not a guarantee of accuracy or compliance for every prompt. Do not use for illegal activity.

Limitations

Quantization can slightly change outputs vs 16-bit
No separate benchmark table shipped with this quant
Inherits limitations of Qwen3-4B and of the DPO dataset

License

Apache 2.0 (same as Qwen base and parent fine-tune).

Citation

bibtex

@misc{zainkhanz_qwen3_4b_toxic_unsloth_bnb_4bit,
  author       = {zainkhanz},
  title        = {Qwen3-4B Toxic Unsloth (4-bit NF4 bitsandbytes)},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {https://huggingface.co/zainkhanz/qwen3-4b-toxic-unsloth-bnb-4bit}
}