Pollard shrank this model: 47.6 GB (f16) → 12.08 GB — 75% smaller, 3.9× down, vs ~12.6 GB at NVFP4.

Full quant ladder below — pick the size that fits your machine.

Carnice on a Mac, not a 3090. Carnice-V3-27b is a 27B agent model (Qwen3.8-27B + a Hermes-agent SFT distilled from Qwen3.8-Max) that punches far above its size on agent tasks — built to fit a 3090 / 24 GB. Pollard sizes it to your box: the recommended build is ~12 GB and runs on a 16 GB Mac.

Built with Pollard Weights on a runtime with native qwen35 support.

Original model: https://huggingface.co/kai-os/Carnice-V3

Model details


Parameter count	27B (dense)
Architecture	`qwen35` (65 layers, 5120 hidden)
Context length	262,144 (256K)
Input support	text + image + video (multimodal)
Specialty	Hermes-agent, tool-use — "beats models 10× the size" on agent tasks
imatrix	yes — Qwen3.8-27B importance matrix (same base; transferable)
Source	Q8_0 (near-lossless; ~99.9% of bf16)

Which file should I choose?

~16 GB Mac / 12 GB VRAM → IQ3_S (12 GB). Runs the 27B where a 3090 was the bar. Recommended headline build.
~24 GB (a 3090) → IQ4_XS (15.7 GB) — smaller than Carnice's own build, more fidelity.
~32 GB → Q6_K (19.6 GB) — near-lossless.

Available files

file	quant	size	held-out PPL	M4 tok/s
Carnice-V3-27b-Pollard-IQ3_S.gguf	IQ3_S mix	12.08 GB	7.88	~6.9
Carnice-V3-27b-Pollard-IQ4_XS.gguf	IQ4_XS mix	15.72 GB	—	24 GB tier
Carnice-V3-27b-Pollard-Q6_K.gguf	Q6_K mix	19.63 GB	—	32 GB tier
mmproj-Carnice-V3-BF16.gguf	vision projector (f16, not quantized)	0.93 GB	—	ship w/ any

(PPL/tok-s measured only on the IQ3 tier — the larger tiers exceed the test Mac's 16 GB, which is the whole point: those are for 24 GB / 32 GB boxes.)

(tok/s is a 16 GB Mac at IQ3 — bandwidth-bound with partial Metal offload; it's markedly faster on 24 GB+ with the whole model resident, and on any CUDA GPU.)

Prompt format

Carnice uses ChatML (Qwen3.5), with tool-use and agent formatting:

<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Multimodal (vision + video)

Ship the mmproj alongside the model (it's kept at f16 — never quantize the projector) and run with llama.cpp's multimodal CLI/server:

bash

llama-mtmd-cli -m Carnice-V3-27b-Pollard-IQ3_S.gguf \
  --mmproj mmproj-Carnice-V3-BF16.gguf --image photo.jpg -p "Describe this image."
# or serve vision:
llama-server -m Carnice-V3-27b-Pollard-IQ3_S.gguf --mmproj mmproj-Carnice-V3-BF16.gguf -ngl 99

Download

bash

pip install -U "huggingface_hub[cli]"
hf download PollardWeights/Carnice-V3-27b-Pollard \
  --include "Carnice-V3-27b-Pollard-IQ3_S.gguf" "mmproj-Carnice-V3-BF16.gguf" --local-dir ./

How to run (text)

bash

curl -LsSf https://llama.app/install.sh | sh
llama-server -hf PollardWeights/Carnice-V3-27b-Pollard:IQ3_S    # OpenAI API + web UI :8080
llama-cli   -m Carnice-V3-27b-Pollard-IQ3_S.gguf -ngl 99 -st -p "Plan a 3-step task."

Needs a recent llama.cpp with qwen35 support. Also runs in LM Studio, koboldcpp, Jan, etc. once they ship a new-enough llama.cpp.

Notes

License: Apache-2.0, inherited from Carnice-V3 / Qwen3.8-27B.
Requantized from Q8_0 (near-lossless source; an f16 source is marginally better, but for an IQ3/IQ4 target the difference is negligible).
imatrix is the base Qwen3.8-27B matrix (Carnice is a merged-LoRA fine-tune of the same base, so it transfers); imatrix-uncovered tensors are pinned to q6_K.
Quantized, not fine-tuned — same weights, sized to your machine.

Credits

Base model: kai-os/Carnice-V3 — Qwen3.8-27B
- Hermes-agent SFT (thanks to NousResearch, LambdaAPI, per the author).
Foundation: Qwen/Qwen3.8-27B.
Quantization runtime: llama.cpp.
Method + tooling: Pollard Weights — measure first, no claim before a number.

PollardWeights/Carnice-V3-27b-Pollard

Pollard quantizations of Carnice-V3-27b by kai-os

Pollard shrank this model: 47.6 GB (f16) → 12.08 GB — 75% smaller, 3.9× down, vs ~12.6 GB at NVFP4.

Model details

Which file should I choose?

Available files

Prompt format

Multimodal (vision + video)

Download

How to run (text)

Notes

Credits

PollardWeights/
Carnice-V3-27b-Pollard