You can deploy this model and start earning money today!

Doses-AI/
Pestle-27B-Ternary-GGUF

Name: Pestle-27B-Ternary-GGUF
Availability: InStock

Views

No views yet

Pestle-27B-Ternary-GGUF

Run locally - Benchmarks - Whitepaper - Model details - Responsible use

A trained 27B ternary medical model: 6.75 GB weight payload, one 8.48 GB runnable GGUF.

8.2× weight compression vs Qwen3.6-27B FP16 at 96.48% median medical capability retention across 17 matched measures · 101.9% of MedGemma-27B’s medical mean · ~10 tok/s on an 18 GB M3 MacBook

vs IQ2_XXS: 10% smaller · ahead or tied on 9/12 shared text-medical metrics · 12% higher shared medical score/GB

Highlights

Full 27B-class model in one 8.48 GB GGUF.
Strong medical performance across clinical knowledge, biomedical QA, and pharmaceutical retrieval.
Runs locally with Mortar on Apple Silicon, NVIDIA CUDA, and CPU fallback.
Text generation, tool-capable chat templates, and optional vision input.
Apache 2.0 licensed.

Benchmarks

Only completed, validated Pestle runs are shown. All locally measured medical runs for Pestle, Qwen3.6-27B FP16, Bonsai-27B Ternary, and Qwen3.6-27B IQ2_XXS used thinking disabled with greedy decoding (temperature=0, top_p=1, top_k=0), no repetition penalty, and seed 20260805. Published external reference cells are identified separately.

Complete comparison matrix

The categorized launch tables and their recorded values are preserved below.

Text medical benchmark comparison

Benchmark	Pestle-27B-Ternary	Qwen3.6-27B FP16	MedGemma 27B FP16	Bonsai 27B Ternary	Qwen3.6-27B IQ2_XXS
BPW (nominal / deployed)	1.79 / 2.52	16.00 / 16.00	16.00 / 16.00	1.71 / 2.13	2.06 / 2.80
Runnable model artifact (GB)	8.48	55.56	54.02	7.17	9.39
Clinical knowledge & reasoning
MedQA	89.79	93.87	89.8 best-of-5 / 87.7 zero-shot (text)	78.24	89.40
MedXpertQA	32.49	41.10	25.7 (text)	20.82	31.22
MedMCQA	68.85	73.70	74.2 (text)	58.59	68.40
MMLU Anatomy	86.67	92.59	83.7 (text)	68.15	84.44
MMLU Clinical Knowledge	90.19	92.45	86.0 (text)	84.53	90.19
MMLU College Biology	96.53	97.92	96.5 (text)	94.44	97.92
MMLU College Medicine	89.60	90.17	86.1 (text)	84.39	88.44
MMLU Medical Genetics	96.00	99.00	97.0 (text)	89.00	98.00
MMLU Professional Medicine	94.85	95.96	93.4 (text)	88.97	93.38
MMLU Virology	56.02	56.02	53.6 (text)	50.60	56.02
ChemBench	61.72	68.73	--	61.33	--
Biomedical QA & Retrieval
PubMedQA (accuracy)	76.70	79.50	77.2 (multimodal)	77.70	78.30
PubMedQA (macro F1)	62.78	64.61	--	62.17	57.74
BioASQ (token F1)	75.28	76.65	--	75.46	--
BioASQ (exact match)	55.94	59.49	--	54.12	--
PharmaRAG (MRR)	90.15	93.91	--	92.60	--
PharmaRAG (nDCG@10)	84.62	90.84	--	89.43	--

_{Size note: Pestle's trained weight payload is 6.75 GB before packaging as one 8,480,707,488-byte runnable GGUF. “Representation BPW” reports the underlying nominal or idealized low-bit representation: Pestle's 1.79-BPW low-bit base before its BF16 overlay, Bonsai's 1.71-BPW ideal ternary representation, and IQ2_XXS's 2.06-BPW quantized block format. “Effective BPW” and decimal GB use runnable or officially downloadable artifacts. Pestle's runnable GGUF equals 2.52 effective BPW over 26,895,998,464 text parameters. Qwen3.6-27B FP16 uses the official 55,562,855,904-byte safetensors payload; MedGemma 27B text FP16 uses 54,018,098,888 bytes across its official shards; Ternary Bonsai uses the downloadable 7,165,121,600-byte Q2_0 GGUF (2.13 deployed BPW); and IQ2_XXS uses the official 9,388,779,744-byte GGUF (2.79 effective BPW, displayed as 2.80).}

Medical intelligence density

_{Medical intelligence density on 10 shared text-only medical benchmarks, defined as D = -log₂(1 - score/100) / downloadable GB. Pestle records 0.275 at 8.48 GB; Qwen3.6-27B IQ2_XXS records 0.245 at 9.39 GB; Bonsai-27B Ternary records 0.234 using a 7.79 GB package comprising its 7.17 GB GGUF and 0.63 GB optional Q8 vision projector; Qwen3.6-27B FP16 records 0.046 at 55.56 GB; and MedGemma-27B FP16 records 0.041 at 54.02 GB. Higher is better.}

_{Metric source: Prism ML's Bonsai.}

General benchmark comparison

Benchmark	Pestle-27B-Ternary	Qwen3.6-27B FP16	Bonsai 27B Ternary	Qwen3.6-27B Q4_K_XL	Qwen3.6-27B IQ2_XXS	Gemma4-31B FP16
Knowledge & reasoning
MMLU-Redux	83.53	93.5	88.05	93.35	88.93	93.60
MuSR	70.50	72.88	65.87	73.01	66.99	71.03
Math
GSM8K	93.25	95.30	96.06	96.66	89.90	97.57
Coding
HumanEval+	87.20	95.12	93.90	95.73	91.46	96.34
MBPP+	83.33	83.33	81.22	83.86	78.89	84.39
Instruction following
IFEval (prompt-strict)	81.70	88.91	85.03	88.83	84.03	90.57

Pestle-27B-Ternary medical and general benchmark landscape

_{Medical-text benchmarks are shown above broader capability context. Solid connections denote locally paired Pestle and Qwen FP16 measurements.}

Experimental vision

Optional vision uses the separately downloaded projection file and is not included in the text-medical headline comparisons.

Benchmark	Pestle-27B-Ternary	Qwen3.6-27B FP16	MedGemma 27B FP16
CXR14 adjudicated (macro F1)	34.28	--	45.3 (multimodal)
CheXpert expert validation (macro F1)	51.03	--	49.9 (multimodal)
VQA-RAD (tokenized F1)	53.49	66.69	46.7 (multimodal)
SLAKE (tokenized F1)	54.81	76.60	70.3 (multimodal)

Deployment performance

Canonical generation protocol

Pestle results use thinking disabled, deterministic greedy decoding (temperature=0, top_p=1, top_k=0), no repetition penalty, and seed 20260805. The locally measured medical Qwen BF16, Bonsai, and IQ2_XXS comparison runs use the same non-thinking sampling protocol. Dataset-specific output ceilings and finish-state handling are documented in the technical report.

Model details

	Pestle-27B-Ternary
Base model	Qwen/Qwen3.6-27B
Architecture	Qwen3.6 (`qwen35` in GGUF metadata)
Parameters	26.9B text parameters
Format	Single-file Pestle ternary GGUF with a BF16 final decoder block
Representation / effective deployed BPW	1.79 / 2.52
Trained weight payload	6.75 GB before GGUF packaging
Runnable text artifact	`pestle-27b-ternary.gguf` (8.48 GB)
Training	Medical-domain optimization and knowledge distillation; one NVIDIA H100, approximately 40 hours
Context length	Up to 262K tokens; practical context depends on available memory
Chat template	Qwen3.6 template, embedded in the GGUF and published as `chat_template.jinja`
Inputs	Text; optional images with `mmproj-pestle-27b-ternary.gguf` (0.93 GB)
Hardware	Apple Silicon via Metal; NVIDIA GPUs via CUDA; CPU fallback
Runtime	Mortar, with a llama.cpp-compatible interface
License	Apache 2.0

The published GGUF is the complete runnable model. No separate overlay assembly or weight conversion is required.

Run locally

1. Download

bash

hf auth login

hf download Doses-AI/Pestle-27B-Ternary-GGUF \
  pestle-27b-ternary.gguf \
  --local-dir models/Pestle-27B-Ternary

For optional vision input, download the matching projection too:

bash

hf download Doses-AI/Pestle-27B-Ternary-GGUF \
  mmproj-pestle-27b-ternary.gguf \
  --local-dir models/Pestle-27B-Ternary

2. Build Mortar

bash

git clone https://github.com/DosesAI/mortar.cpp.git
cd mortar.cpp
./scripts/build-mortar.sh

The build script selects Metal on macOS, CUDA on Linux when a compatible NVIDIA GPU is visible, and CPU otherwise.

For a CPU-only build on macOS or Linux:

bash

./scripts/build-mortar.sh --cpu
./mortar --cpu-only \
  --model ../models/Pestle-27B-Ternary/pestle-27b-ternary.gguf

On Windows PowerShell:

powershell

.\scripts\build-mortar.ps1 -Backend cpu
.\mortar.ps1 --cpu-only `
  --model ..\models\Pestle-27B-Ternary\pestle-27b-ternary.gguf

CPU-only inference uses the same GGUF without conversion. It requires enough system RAM for the model and context and is substantially slower than Metal or CUDA inference.

3. Chat

bash

./mortar \
  --model ../models/Pestle-27B-Ternary/pestle-27b-ternary.gguf

Or start with a prompt:

bash

./mortar \
  --model ../models/Pestle-27B-Ternary/pestle-27b-ternary.gguf \
  --prompt "Explain the mechanism of action of metformin."

Mortar applies the validated Pestle runtime and deterministic defaults automatically.

Recommended generation settings

Setting	Value used for reported evaluations
Thinking	Off
Temperature	0
Top-p	1
Top-k	0
Repetition penalty	1
Seed	20260805

The model supports thinking mode; enable it only when desired and treat it as a different evaluation/deployment configuration.

Architecture

Pestle is a trained ternary medical model developed through Doses AI’s medical-domain optimization and knowledge distillation. It preserves the Qwen3.6-27B hybrid-attention architecture while applying Doses AI ternary compression across the language model. The release combines a compact ternary core with a matching-parent BF16 final decoder block and ships as a single runnable GGUF.

Optional vision input uses the separately downloadable matching projection file.

Intended use

Pestle is a research preview intended for evaluation and development of locally hosted medical-text, biomedical retrieval, pharmaceutical, coding, and general assistant systems. It is not released for clinical use.

Responsible use

Pestle is a research preview for evaluation and development. It is not a medical device and is not intended to make diagnosis, treatment, prescribing, triage, or patient-management decisions.

Outputs may be inaccurate, incomplete, biased, or confidently wrong.
Medical outputs require review by qualified professionals and validation for the intended setting.
Do not send identifiable patient information to an environment that is not approved to process it.
Local execution can support privacy and data-residency goals, but does not by itself establish regulatory compliance.
Long outputs can occasionally become repetitive; production systems should use sensible generation limits and monitoring.

Citation

bibtex

@misc{dosesai2026pestle,
  title        = {Pestle-27B-Ternary},
  author       = {{Doses AI}},
  year         = {2026},
  howpublished = {Hugging Face model release},
  url          = {https://huggingface.co/Doses-AI/Pestle-27B-Ternary-GGUF}
}

Links

Built by **Doses AI**