Self-quantized AWQ 4-bit checkpoint of
mistralai/Mistral-7B-Instruct-v0.3 with fully documented calibration provenance.
Pre-quantized checkpoints on HuggingFace typically have unknown calibration provenance — the dataset, sample count, seed, and group size are rarely documented. This checkpoint was self-quantized with controlled, documented settings to enable rigorous cross-method comparison (GGUF k-quant vs AWQ vs GPTQ) in a NeurIPS 2026 submission on quality-safety correlation under quantization.
Evaluation pending — quality and safety benchmarks will be run on this checkpoint and results updated here.
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model = AutoModelForCausalLM.from_pretrained(
4 "Crusadersk/mistral-7b-awq-4bit",
5 device_map="auto",
6)
7tokenizer = AutoTokenizer.from_pretrained("Crusadersk/mistral-7b-awq-4bit")
8
9messages = [{"role": "user", "content": "What is the capital of France?"}]
10inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
11outputs = model.generate(inputs, max_new_tokens=64, do_sample=False)
12print(tokenizer.decode(outputs[0], skip_special_tokens=True))
The full quantization pipeline — Dockerfiles, quantization scripts, and a 766-line engineering log documenting every platform failure and solution — is available at:
1@misc{banterhearts2026mistral7bawq,
2 title = {Self-Quantized Mistral 7B Instruct v0.3 (AWQ 4-bit) for Quality-Safety Correlation Research},
3 author = {Kadadekar, Sahil},
4 year = {2026},
5 url = {https://huggingface.co/Crusadersk/mistral-7b-awq-4bit},
6 note = {Part of the Banterhearts research program. NeurIPS 2026 submission.}
7}
This work is part of the Chimera/Banterhearts technical-report program on deployment-time LLM behavior, quantization, refusal robustness, batching effects, and inference-stack reliability. Canonical public archive:
Chimeraforge Reports; source context:
github.com/Sahil170595/Banterhearts.
