🔬 This repository is part of a production-oriented evaluation series. Every model published under pbhappliedsystems has been independently evaluated using quant_eval v7.21 — a proprietary behavioral evaluation harness developed by PBH Applied Systems.

📌 This is the full-precision F16 baseline repository. The evaluated Q4_K_M deployment variant is published at pbhappliedsystems/qwen-2.5-3B-instruct-gguf-Q4-K-M. That card documents the full F16 vs. Q4_K_M comparison, including the ms_easy_01 schema recovery, mixed_brief_json recovery (0.000 → 1.000), and the runner-specific output format differences documented below.

⚖️ License Notice: This model is governed by the Qwen Research License, which permits non-commercial use only. Commercial use requires a separate license from Alibaba Cloud. See the License section for full details.

Try the Live AI Agent Demo

Launch the PBH Applied Systems AI Agent Demo →

This model is part of the PBH Applied Systems evaluated model series that supports the live AI Agent Demo. The demo lets visitors interact with production-style agent workflows powered by open-weight language models evaluated through PBH Applied Systems' quant_eval framework.

The F16 model serves a different role than the Q4_K_M deployment variant. F16 is the full-precision baseline used to measure what the model can do before quantization. quant_eval then compares the quantized model against this baseline to identify which capabilities are preserved, which degrade, and which tasks require guardrails or a higher-precision deployment.

This comparison is central to the demo. It helps determine which model belongs in which agent role:

Reasoning models are selected for planning, analysis, and auditable decision workflows.
Document models are selected for long-context extraction, summarization, and structured Q&A.
Code models are selected for task completion, structured output, API scaffolding, and automation workflows.
Quantized variants are selected when they preserve enough behavior to reduce cost, latency, and GPU requirements.
F16 variants remain important when maximum fidelity, cleaner tool execution, or reduced quantization risk matters more than speed or cost.

The live demo shows the deployment side of that process. The F16 card documents the reference behavior. The Q4_K_M card shows what changes after compression. Together, they explain how PBH Applied Systems uses quant_eval to choose the correct LLM for the correct agent type instead of guessing from model size or leaderboard reputation.

Model Description

This repository contains the full-precision F16 GGUF of Qwen/Qwen2.5-3B-Instruct, a 3-billion parameter instruction-tuned model from Alibaba Cloud.

In the PBH Applied Systems evaluation pipeline, this F16 run (20260221_034857) operated in cache-generation mode (skip_quant=true), producing the full_weight_cache.json used as the reference baseline for the subsequent Q4_K_M comparison run (20260221_041137). The F16 evaluation results documented here are identical to the F16 baseline data shown in the Q4_K_M card — confirmed by matching timing profiles, identical family results, and the same 5 failing fuzz cases on both runs.

Key Characteristics

Parameters: 3B
Format: GGUF F16 (full precision)
File size: 6.18 GB
SHA256: 65a0239fc9f9a40e2d4f79ae5e158cad423c2476fe089c744d5e6a6ff6fc9330
Minimum VRAM (GPU inference): ~8 GB
Recommended GPU tier: RTX 3060 · T4 · Any 8 GB+ GPU
Context window: 32,768 tokens
Inference speed (eval hardware): avg 1.438 sec/case on RTX 4090
License: Qwen Research License (non-commercial)

On F16 vs. Q4_K_M inference speed: The F16 averages 1.438 sec/case vs 0.390 sec/case for Q4_K_M — a 3.7× speedup at quantization. Both are fast by series standards. The Q4_K_M card notes MCQ at 10ms and stateful followup at 125ms — performance that remains broadly available at F16 too given the 3B parameter count.

PBH Applied Systems Evaluation — quant_eval v7.21

Evaluation conducted by PBH Applied Systems, LLC using quant_eval v7.21 Run ID: 20260221_034857 · Fixtures: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c...) · Seed: 42 Hardware: NVIDIA RTX 4090 · Runner: full_weight_transformers (F16 only) · Total rows: 42

Per-Family Pass Rates — F16 (`full_weight_transformers`)

Family	N	Pass Rate	Avg Secs	Bucket Score	Notes
json_multistep	5	0.000	3.704	1.000	checks_consistent_ok bottleneck — see Q4_K_M for comparison
stateful_followup	2	1.000	0.525	2.000	Both turns exact match
toolcall_only	2	0.000	0.405	1.000	tool_name_ok=1, args schema incorrect
mixed_brief_json	2	0.000	0.695	1.000	JSON valid; ANSWER line stripped — see note
toolcall	2	1.000	0.745	0.000	Stage-1 passes; no final answer emitted — see note
json	4	n/a	2.072	10.000	All pass
fuzz	20	n/a	1.437	7.500	Same 15/20 as Q4_K_M
mcq	5	n/a	0.022	0.000	Empty output on all 5

json_multistep — Case-Level Breakdown

Case	Difficulty	Result	Secs	Failure Signals
ms_easy_01	Easy	❌ FAIL	2.534	schema_ok=0 (array output — see below)
ms_easy_02	Easy	❌ FAIL	3.504	cc=0, stop=0
ms_med_01	Medium	❌ FAIL	4.514	oracle_equiv_ok=0 only
ms_med_02	Medium	❌ FAIL	4.032	cc=0, stop=0, oe=0
ms_hard_01	Hard	❌ FAIL	3.936	cc=0, stop=0, oe=0

No case passes at F16. The Q4_K_M variant recovers ms_easy_01 (1.000 on all gating signals). The checks_consistent_ok signal is the persistent blocker — 0.200 at both precision levels, confirming this is a 3B parameter count limitation.

F16-Specific Findings

Finding 1: Role-Token Contamination in Outputs

The HuggingFace Transformers runner includes truncated chat role tokens as literal text in several response families. Raw output inspection reveals:

Family	Raw Output Pattern	Effect
stateful_followup	`ician {"counter": 2} ician {"counter": 5}`	"ician" prefix (truncated "technician") — extraction still works because JSON follows
toolcall	`ician {"tool_name": "add", "args": {...}} user`	Role tokens surround tool call; no answer number present
mixed_brief_json	`user ANSWER: 13 {"a": 4, "b": 9, "sum": 13}`	"user" prefix before ANSWER line — extractor cannot find `ANSWER:` at line start
json	`user {"tool_name": "place_item", "args": {...}}`	"user" prefix; JSON extraction handles this — all 4 pass

The stateful_followup family passes despite the contamination because the JSON state objects are extracted correctly regardless of the "ician" prefix. The json family passes for the same reason. mixed_brief_json fails because answer_line_ok specifically requires the ANSWER: prefix to appear at the start of a line — the "user" token breaks this pattern.

Finding 2: toolcall — Valid Dispatch, No Final Answer

At F16, toolcall achieves stage1_tool_parse_ok=1 and stage1_tool_schema_ok=1 on both cases — the tool call JSON is correctly formed and schema-valid. However, the final answer number is never emitted:

Case	F16 Raw	Expected Final	Result
tool_01	`ician {"tool_name": "add", "args": {"a": 2, "b": 3}} user`	`5`	❌ final_mismatch
tool_02	`ician {"tool_name": "add", "args": {"a": 10, "b": -4}} user`	`6`	❌ final_mismatch

The model produces the tool call and then terminates with a "user" role token, never computing or reporting the result. Contrast with Q4_K_M where the same cases produce {tool call}<|im_end|> 5<|im_end|> — the arithmetic result is present but EOS-contaminated. At F16, the result is absent entirely. Both are final_mismatch, but for different reasons: F16 = no answer produced; Q4_K_M = correct answer with stop-token contamination.

Finding 3: ms_easy_01 — Wrong Output Type at F16

ms_easy_01 at F16 produces an array instead of the required schema object:

[{"shelf": "A", "item": "P", "can_place": 1}]

Expected format: {"plan": [...], "checks": [...], "final": {...}}. This schema_ok=0 failure is unique in the 3B evaluation — no other multistep case produces an entirely wrong output type. At Q4_K_M, this same case passes cleanly at 0.785 seconds with all signals correct.

Finding 4: toolcall_only — Args Format Difference Between Runners

Both runners fail toolcall_only on args_ok, but with slightly different incorrect schemas:

Runner	Raw Output	Why args_ok=0
F16	`{"tool": "add", "args": [5, 10]}`	`args` is array; expected `{"a": 5, "b": 10}` object
Q4_K_M	`{"tool": "add", "operands": [5, 10]}`	Wrong key (`operands`); value is array

Both use "tool" instead of "tool_name" as the outer key (non-gating schema issue). Both provide numeric values as an array rather than named keys. The underlying 3B model behavior is the same at both precision levels — it cannot resolve the exact required schema for bare tool dispatch without explicit key-name enforcement in the prompt.

Finding 5: MCQ — Empty Output at Both Precision Levels

All 5 MCQ cases produce invalid_choice raw='' at F16 in under 30 milliseconds. No visible output is emitted. The Q4_K_M runner produces single letter outputs (B, A, C) on the same cases, with 3/5 correct. The F16 Transformers runner does not surface the choice letter in a form the extractor can process.

Signal-Level Diagnostics (F16)

json_multistep

Signal	F16 Rate	Q4_K_M Rate	Delta
schema_ok	0.800	1.000	+0.200
checks_consistent_ok	0.200	0.200	0.000
stop_semantics_ok	0.200	0.400	+0.200
oracle_equiv_ok	0.200	0.400	+0.200

The checks_consistent_ok signal is invariant across precision levels — it reflects the 3B model's reasoning consistency ceiling, not quantization effects. Every other signal improves at Q4_K_M.

stateful_followup

Signal	Rate
turn1_parse_ok	1.000
turn2_parse_ok	1.000
turn1_exact_match	1.000
turn2_exact_match	1.000

toolcall_only

Signal	Rate
tool_name_ok	1.000
args_ok	0.000

mixed_brief_json

Signal	Rate
answer_line_ok	0.000
json_parse_ok	1.000
schema_ok	1.000

When to Deploy F16 vs. Q4_K_M

Criterion	F16 (this repo)	Q4_K_M
VRAM required	~8 GB	~4 GB
Avg inference time	1.438 sec/case	0.390 sec/case
json_multistep pass rate	0.000	0.200
ms_easy_01	❌ Wrong output type	✅ PASS
mixed_brief_json	❌ Role-token blocks ANSWER line	✅ 1.000
toolcall final answer	❌ Not emitted	⚠️ Present but EOS-contaminated
toolcall stage-1	✅ 1.000	✅ 1.000
MCQ extraction	❌ Empty output	⚠️ 3/5 pass
stateful_followup	✅ 1.000	✅ 1.000
json / fuzz	✅ 10.000 / 7.500	✅ 10.000 / 7.500

For this model, Q4_K_M is the stronger deployment choice across every measurable family. The F16 runner exhibits role-token contamination that degrades families the Q4_K_M runner handles cleanly. The only advantage of F16 is slightly higher VRAM availability allowing larger context windows.

Hardware Requirements

Configuration	VRAM Required	Notes
F16 (this repo) · GPU	~8 GB	6.18 GB model + KV cache overhead
F16 · partial CPU offload	4–6 GB VRAM + 4 GB RAM	Viable for reduced context
Q4_K_M (companion repo)	~4 GB	1.93 GB — fits on edge hardware

Usage

Installation

pip install llama-cpp-python huggingface_hub

For GPU acceleration (CUDA):

CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python --force-reinstall --no-cache-dir

Python — llama-cpp-python

python

from huggingface_hub import hf_hub_download
from llama_cpp import Llama

model_path = hf_hub_download(
    repo_id="pbhappliedsystems/qwen-2.5-3B-instruct-gguf-F16",
    filename="qwen-2.5-3B-instruct-gguf-F16.gguf"
)

llm = Llama(
    model_path=model_path,
    n_ctx=4096,
    n_gpu_layers=-1,
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": "You are a precise assistant. Return structured JSON when asked. Use exactly the key names specified."
        },
        {
            "role": "user",
            "content": "Return a JSON object with keys: summary, sentiment, action_items."
        }
    ],
    temperature=0.15,
    max_tokens=512,
)

print(response["choices"][0]["message"]["content"])

For stateful multi-turn use (reliable at F16 — role-token prefix does not block JSON extraction):

python

import json, re

conversation = [
    {"role": "system", "content": "You are a stateful assistant tracking structured data."},
    {"role": "user", "content": "Initialize a counter at 1. Return JSON only: {\"counter\": N}"},
]

r1 = llm.create_chat_completion(messages=conversation, temperature=0.15, max_tokens=64)
t1 = r1["choices"][0]["message"]["content"]
# Strip role-token prefix if present
t1_clean = re.sub(r'^[a-z]+\s*', '', t1.strip())

conversation.append({"role": "assistant", "content": t1_clean})
conversation.append({"role": "user", "content": "Increment the counter by 1."})

r2 = llm.create_chat_completion(messages=conversation, temperature=0.15, max_tokens=64)
t2_clean = re.sub(r'^[a-z]+\s*', '', r2["choices"][0]["message"]["content"].strip())
print(json.loads(t2_clean))

CLI — llama-cli

bash

llama-cli \
  --model qwen-2.5-3B-instruct-gguf-F16.gguf \
  --chat-template qwen2 \
  --system-prompt "You are a precise assistant. Use exactly the key names specified." \
  --prompt "Return a JSON object with keys: summary, risk_level, action_items." \
  --n-predict 512 \
  --ctx-size 4096 \
  --n-gpu-layers -1 \
  --temp 0.15

For server deployment:

bash

llama-server \
  --model qwen-2.5-3B-instruct-gguf-F16.gguf \
  --chat-template qwen2 \
  --ctx-size 4096 \
  --n-gpu-layers -1 \
  --port 8080 \
  --host 0.0.0.0

Query via the OpenAI-compatible API:

python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-required")

response = client.chat.completions.create(
    model="qwen-2.5-3B-instruct-gguf-F16",
    messages=[{"role": "user", "content": "Your prompt here"}],
    temperature=0.15,
)
print(response.choices[0].message.content)

Artifact Provenance

Artifact	Format	Size	SHA256
`qwen-2.5-3B-instruct-gguf-F16.gguf`	GGUF F16	6.18 GB	`65a0239fc9f9a40e2d4f79ae5e158cad423c2476fe089c744d5e6a6ff6fc9330`
Q4_K_M (companion repo)	GGUF Q4_K_M	1.93 GB	`9ab3bc9beaddaec3700d5cc754b52e1501a3fd172bc7fc3ee3eb8e1d388ee043`

The F16 GGUF was converted from Qwen/Qwen2.5-3B-Instruct using a custom-built llama.cpp conversion pipeline developed by PBH Applied Systems.

Two-pass architecture: This F16 run (20260221_034857) operated in cache-generation mode (skip_quant=true), producing the full_weight_cache.json used as the reference baseline for the Q4_K_M comparison run (20260221_041137). Timing profile identity between this run and the F16 baseline in the comparison run confirms clean cache reuse and run integrity.

Evaluation Methodology

quant_eval v7.21 — proprietary behavioral evaluation harness, PBH Applied Systems.

Fixture set: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c079371b02a94f3c149eb78a6adc03dc16ff6833b964fbf4174f0)

Evaluation hardware: NVIDIA RTX 4090 · F16 evaluation date: February 21, 2026 · Seed: 42

🔬 About quant_eval & This Evaluation Series

quant_eval is a proprietary behavioral evaluation harness developed by PBH Applied Systems, LLC. It measures real agent-adjacent task performance across structured output, tool dispatch, multi-turn state retention, and multi-step planning — not perplexity or leaderboard proxies. Every model published under pbhappliedsystems has been independently evaluated using quant_eval before being recommended for any production role.

See it in action: Live AI Agent Demo → The demo runs production-style agent workflows powered by open-weight models selected through the quant_eval evaluation pipeline.

Need a deployment recommendation? Not sure which quantization level is right for your hardware, latency target, or agent type? → pbhappliedsystems.com

Evaluated and published by PBH Applied Systems, LLC · patrick@pbhappliedsystems.com

About PBH Applied Systems

PBH Applied Systems, LLC is an Oklahoma City–based applied machine learning and AI systems company specializing in production-grade model evaluation, quantization pipelines, agentic AI infrastructure, and scalable AI-driven application development.

Patrick Hill, M.S. — Founder · Data Scientist · AI/ML Engineer · Author of Applied Machine Learning: Concepts, Tools, and Case Studies (required reading, UAT CSC 373)

📞 Work With PBH Applied Systems

The F16 vs. Q4_K_M comparison for this model documents that the Q4_K_M runner handles role tokens and stop tokens more cleanly than the Transformers F16 runner — resulting in better practical outcomes at lower VRAM requirements. That counter-intuitive finding is only visible when both variants are evaluated against the same fixture set.

👉 Book a Scoping Call · 👉 Request an Evaluation Report — from $2,500

Connect


🌐	pbhappliedsystems.com
📧	patrick@pbhappliedsystems.com
💼	LinkedIn
▶️	YouTube
📸	Instagram
👍	Facebook

License

Qwen Research License Agreement — non-commercial use only. Inherited from Qwen/Qwen2.5-3B-Instruct.

Commercial use requires a separate license from Alibaba Cloud. Full terms: Qwen Research License

GGUF conversion and behavioral evaluation performed by PBH Applied Systems, LLC · quant_eval v7.21 · F16 Run ID: 20260221_034857

pbhappliedsystems/qwen-2.5-3B-instruct-gguf-F16

Qwen2.5-3B-Instruct · GGUF F16

Try the Live AI Agent Demo

Model Description

Key Characteristics

PBH Applied Systems Evaluation — quant_eval v7.21

Per-Family Pass Rates — F16 (full_weight_transformers)

json_multistep — Case-Level Breakdown

F16-Specific Findings

Finding 1: Role-Token Contamination in Outputs

Finding 2: toolcall — Valid Dispatch, No Final Answer

Finding 3: ms_easy_01 — Wrong Output Type at F16

Finding 4: toolcall_only — Args Format Difference Between Runners

Finding 5: MCQ — Empty Output at Both Precision Levels

Signal-Level Diagnostics (F16)

json_multistep

stateful_followup

toolcall_only

mixed_brief_json

When to Deploy F16 vs. Q4_K_M

Hardware Requirements

Usage

Installation

Python — llama-cpp-python

CLI — llama-cli

Artifact Provenance

Evaluation Methodology

🔬 About quant_eval & This Evaluation Series

About PBH Applied Systems

📞 Work With PBH Applied Systems

Connect

License

pbhappliedsystems/
qwen-2.5-3B-instruct-gguf-F16

Per-Family Pass Rates — F16 (`full_weight_transformers`)