Quantized, converted, and evaluated by PBH Applied Systems, LLC — Applied AI/ML Consulting · LLM Optimization & Deployment · Quantized AI Infrastructure

🔬 This repository is part of a production-oriented evaluation series. Every model published under pbhappliedsystems has been independently evaluated using quant_eval v7.21 — a proprietary behavioral evaluation harness developed by PBH Applied Systems. Scores measure real agent-adjacent task performance across structured output, tool dispatch, multi-turn state retention, and multi-step planning families — not perplexity or benchmark leaderboard proxies.

Try This Model in the Live AI Agent Demo

Launch the PBH Applied Systems AI Agent Demo →

This model is part of the PBH Applied Systems live AI Agent Demo, where visitors can test evaluated quantized open-weight models across production-style agent workflows: reasoning and analysis, document intelligence, and code automation.

The demo uses quant_eval results to show how model selection changes by task. A model that performs well for long-context document analysis may not be the best choice for hard multi-step planning, strict tool-use workflows, or production code generation. Each deployed model is evaluated for practical agent behavior, including coherence, instruction following, reasoning, task completion, structured output reliability, tool-use behavior, and quantization impact.

For this repository, the Q4_K_M variant represents the deployment-focused model: smaller, faster, and more cost-efficient than the F16 baseline. The evaluation results below explain where this quantized model preserves useful behavior, where quantization introduces risk, and what guardrails are recommended before production deployment.

The purpose of the demo is simple: let prospects test the same kind of evaluated quantized models that PBH Applied Systems deploys for real agentic AI systems.

Model Description

This repository contains the 4-bit quantized (Q4_K_M) GGUF of mistralai/Mistral-Nemo-Instruct-2407, a 12-billion parameter instruction-tuned model developed by Mistral AI in collaboration with NVIDIA (July 2024 release). Mistral-Nemo features the Tekken tokenizer and a 128,000-token context window — the largest context in the evaluated series, outside of the Qwen2.5-14B-Instruct-1M, which supports a 1 million token window, in the PBH Applied Systems evaluated series.

The Q4_K_M format applies 4-bit quantization with K-quant medium precision. As documented in the evaluation section below, Q4_K_M quantization produces measurable degradation on multi-step planning tasks — including a complete breakdown on the hardest planning case — while preserving strong performance on stateful, structured-output, and hybrid response tasks.

The full-precision F16 baseline is published separately at pbhappliedsystems/mistral-nemo-instruct-2407-gguf-F16.

Key Characteristics

Parameters: 12B
Format: GGUF Q4_K_M
File size: 7.48 GB
SHA256: 5765024ff3361f6dc5b590b963b378bd2e87ac95eabe5823a08a3ad336b498c9
Context window: 128,000 tokens (Tekken tokenizer)
Minimum VRAM (GPU inference): ~10 GB (T4 class or better)
Recommended GPU tier: NVIDIA T4 (16 GB) · RTX 3080/4080 · A10G
Inference speed (eval hardware): avg 1.42 sec/case on RTX 4090
Multilingual: English, French, German, Spanish, Italian, Portuguese, Russian, Chinese, Japanese

PBH Applied Systems Evaluation — quant_eval v7.21

Evaluation conducted by PBH Applied Systems, LLC using quant_eval v7.21 Run ID: 20260211_022944 · Fixtures: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c...) · Seed: 42 Hardware: NVIDIA RTX 4090 · Total rows evaluated: 84 (42 F16 · 42 Q4_K_M)

Aggregate Scores (Q4_K_M)

Scores are normalized to [0.0 – 1.0]. Higher is better.

Dimension	Score
Task Completion	0.6631
Reasoning	0.7870
Coherence	0.8836
Instruction Following	0.9329
Avg inference time	1.42 sec/case

Per-Family Pass Rates

F16 Baseline (`full_weight_transformers`)

Family	N	Pass Rate	Avg Secs	Notes
json_multistep	5	0.600	79.08	ms_easy_02 + ms_hard_01 fail
stateful_followup	2	1.000	7.60	Both turns exact match
toolcall_only	2	1.000*	9.54	Gating passed; schema wrapper issue — see note
mixed_brief_json	2	1.000	10.59	Answer line + JSON schema correct
toolcall	2	1.000	14.39	Tool parse + schema valid
json	4	n/a	64.02	bucket_score avg = 10.000 (json_01 = 149.88s outlier)
fuzz	20	n/a	26.61	bucket_score avg = 10.000
mcq	5	n/a	0.46	bucket_score avg = 0.600 — 2 failures

Q4_K_M (`quantized_llama_cpp`)

Family	N	Pass Rate	Δ vs F16	Avg Secs	Notes
json_multistep	5	0.400	−0.200	4.52	⚠️ Hard case complete breakdown
stateful_followup	2	1.000	0.000	0.40	Perfect retention
toolcall_only	2	0.000	−1.000	0.46	tool_name_ok=1, args_ok=0
mixed_brief_json	2	1.000	0.000	0.52	No degradation
toolcall	2	1.000	0.000	0.62	Pass rate holds; final_mismatch on tool_02 — see note
json	4	n/a	—	1.65	bucket_score avg = 10.000
fuzz	20	n/a	—	1.31	bucket_score avg = 10.000
mcq	5	n/a	—	0.03	bucket_score avg = 0.400 — 3 failures

Key Findings

Finding 1: json_multistep — Complete Breakdown on Hard Case

The drop from 0.600 (F16) to 0.400 (Q4_K_M) is the largest json_multistep degradation in the PBH Applied Systems evaluated series to date. The case-level breakdown reveals why:

Case	Difficulty	F16 Result	Q4_K_M Result	Q4_K_M Signals
ms_easy_01	Easy	✅ PASS	✅ PASS	All pass
ms_easy_02	Easy	❌ FAIL	❌ FAIL	cc=0, oe=0
ms_med_01	Medium	✅ PASS	✅ PASS	All pass
ms_med_02	Medium	✅ PASS	❌ FAIL	cc=0 only
ms_hard_01	Hard	❌ FAIL	❌ FAIL	ALL 4 signals fail

ms_hard_01 at Q4_K_M is a total failure: schema_ok=0, checks_consistent_ok=0, stop_semantics_ok=0, oracle_equiv_ok=0. All four Tier-1 gating signals fail simultaneously. The model does not produce a parseable schema response, its intermediate checks are self-inconsistent, its STOP semantics are wrong, and the computed final state does not match the oracle. The F16 variant fails this case too — but only on consistency and oracle, not on schema or STOP semantics. Quantization turns a partial failure into a complete one.

ms_med_02 is a new failure at Q4_K_M that passes at F16: checks_consistent_ok=0 with oracle_equiv_ok=1 — the model arrives at the correct final state but its internal reasoning steps are self-inconsistent. This is a structural coherence regression under quantization.

Practical implication: This model at Q4_K_M should not be used for multi-step planning tasks without an external validation layer. The hard case cannot be considered reliably solvable at this precision level.

Finding 2: toolcall — Final Mismatch on tool_02 (Q4_K_M)

toolcall passes at 1.000 (both stage1 signals pass), but tool_02 shows detail=final_mismatch with bucket_score=0 at Q4_K_M. The tool call JSON is dispatched and validated correctly — the stage-1 parse and schema check both pass — but the model's final answer (the computed result returned after tool execution) does not match the expected output.

Case	F16 bucket	Q4_K_M bucket	Q4_K_M detail
tool_01	11	11	ok
tool_02	11	0	final_mismatch

This is not a gating failure — the pass rate remains 1.000 because Tier-1 only evaluates the dispatch quality, not the final answer. However, in a production pipeline where tool results feed downstream computation, a final_mismatch means the model called the tool correctly but gave a wrong answer when reporting the result. For applications where post-tool reasoning accuracy matters, treat this as a deployment risk at Q4_K_M.

The Q4_K_M toolcall bucket_score average of 5.5 (vs 11.0 at F16) directly reflects this: one perfect (11) and one complete failure (0) averaged together.

Finding 3: toolcall_only — Consistent args Failure with Stable Tool Name

Signal	F16 Rate	Q4_K_M Rate
tool_name_ok	1.000	1.000
args_ok	1.000	0.000
schema_ok	0.000*	0.000

At F16, toolcall_only passes gating (tool_name_ok=1, args_ok=1) but carries the same schema wrapper non-compliance observed across multiple models in this series (schema_ok=0, detail=schema_error). At Q4_K_M, tool_name_ok stays perfect at 1.000 — the model correctly identifies which tool to call — but args_ok drops to 0.000 on both cases. The quantized model knows the tool name but cannot construct a valid argument payload.

*F16 schema_ok=0 is a non-gating wrapper issue (uses "tool" instead of "tool_name" as outer key), not a capability failure. Both gating signals pass at F16.

Finding 4: MCQ "got=A" Bias

Both runners show a systematic bias toward selecting choice A on failures:

Case	F16 result	Q4_K_M result
mcq_01	✅ ok	✅ ok
mcq_02	❌ wrong_choice got=A	❌ wrong_choice got=A
mcq_03	✅ ok	✅ ok
mcq_04	✅ ok	❌ wrong_choice got=A
mcq_05	❌ wrong_choice got=A	❌ wrong_choice got=A

Every failure on both runners produces got=A. This is a model-level characteristic: when uncertain, this model defaults to option A. Q4_K_M extends this bias to mcq_04 (which F16 answers correctly), reducing the bucket_score from 0.600 to 0.400. For MCQ applications, be aware of this A-default tendency and consider instruction-tuning or chain-of-thought prompting to elicit more deliberate choice selection.

Signal-Level Diagnostics (Q4_K_M)

json_multistep

Signal	F16 Rate	Q4_K_M Rate	Delta
schema_ok	1.000	0.800	−0.200
checks_consistent_ok	0.800	0.400	−0.400
stop_semantics_ok	1.000	0.800	−0.200
oracle_equiv_ok	0.600	0.600	0.000
final_consistent_ok	0.000	0.000	0.000
final_match_reported	0.000	0.000	0.000

checks_consistent_ok takes the largest hit (−0.400), dropping from 0.800 to 0.400. This signal measures whether the model's intermediate reasoning steps are internally self-consistent. The Q4_K_M degradation here is the root cause of the json_multistep pass rate drop: the model fails ms_med_02 on consistency alone, and ms_hard_01 on all signals.

stateful_followup

Signal	Rate
turn1_parse_ok	1.000
turn2_parse_ok	1.000
turn1_exact_match	1.000
turn2_exact_match	1.000

toolcall_only (Q4_K_M)

Signal	Rate
tool_name_ok	1.000
args_ok	0.000

mixed_brief_json

Signal	Rate
answer_line_ok	1.000
json_parse_ok	1.000
schema_ok	1.000

Recommended Use Cases

✅ Deploy with Confidence (Q4_K_M)

Stateful multi-turn agents — Perfect two-turn state retention (1.000). Reliable at 0.40 sec/case.
Structured JSON outputs (single-step) — json and fuzz both achieve bucket_score 10.000. Valid constraint-adherent outputs every case.
Hybrid brief + JSON responses — mixed_brief_json passes at 1.000. Fast at 0.52 sec/case.
Long-context document processing — 128K token context window is the key differentiator for this model. Suitable for full-document Q&A, multi-document comparison, and long-form extraction tasks.
Multilingual structured tasks — Trained on 9 languages. Reliable for non-English structured output pipelines.
Tool-calling with response scaffolding — toolcall pass rate holds at 1.000. Use stage-1 tool dispatch reliably; add final-answer validation for downstream computation (see tool_02 finding).

⚠️ Use with Guardrails (Q4_K_M)

Multi-step planning at easy-to-medium difficulty — ms_easy_01 and ms_med_01 pass. ms_med_02 fails on internal consistency. Use with an external validation loop for any planning task beyond trivial difficulty.
Post-tool answer validation — toolcall dispatches correctly but tool_02 returns a wrong final answer. Validate model output after tool execution, not just the tool call itself.
Bare tool-call dispatch — toolcall_only fails on args (0.000). Add a schema enforcement layer or use scaffolded tool calling.

❌ Not Recommended (Q4_K_M)

Hard multi-step planning — ms_hard_01 fails on all four gating signals simultaneously. Do not deploy for hard planning tasks without F16 or an external planner/verifier.
MCQ without A-bias mitigation — Three of five MCQ cases fail at Q4_K_M, all defaulting to A. Add chain-of-thought prompting or answer validation for MCQ-style pipelines.

128K Context Window — Deployment Considerations

Mistral-Nemo's 128K context window is a meaningful production advantage for document-intensive applications. At Q4_K_M (7.48 GB model weight), actual usable context depends on KV cache VRAM overhead:

Context Length	Approx. KV Cache	Total VRAM Needed	Fits on
8K tokens	~0.5 GB	~10 GB	T4 16 GB
32K tokens	~2 GB	~12 GB	T4 16 GB · RTX 4080
64K tokens	~4 GB	~14 GB	A10G 24 GB · RTX 4090
128K tokens	~8 GB	~18 GB	A10G 24 GB · RTX 4090

Set n_ctx in llama-cpp-python to the actual context length you need — do not default to 128K if your use case only needs 8K. Unnecessary context allocation wastes VRAM and slows inference.

Hardware Requirements

Configuration	VRAM Required	Recommended GPU
Q4_K_M · 8K context (this repo)	~10 GB	T4 16 GB · RTX 3080
Q4_K_M · 128K context	~18 GB	A10G 24 GB · RTX 4090
F16 baseline (companion repo)	~26 GB	A100 40 GB · RTX 4090 · 2× A10G

Usage

Installation

pip install llama-cpp-python huggingface_hub

For GPU acceleration (CUDA):

CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python --force-reinstall --no-cache-dir

Python — llama-cpp-python

python

from huggingface_hub import hf_hub_download
from llama_cpp import Llama

model_path = hf_hub_download(
    repo_id="pbhappliedsystems/mistral-nemo-instruct-2407-gguf-Q4-K-M",
    filename="mistral-nemo-instruct-2407-gguf-Q4-K-M.gguf"
)

llm = Llama(
    model_path=model_path,
    n_ctx=32768,      # Adjust to your use case; model supports up to 128K
    n_gpu_layers=-1,  # -1 offloads all layers to GPU
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": "You are a precise assistant. Follow instructions exactly and return structured outputs when requested."
        },
        {
            "role": "user",
            "content": "Analyze the following document and return a JSON object with keys: summary, key_entities, sentiment, action_items."
        }
    ],
    temperature=0.3,
    max_tokens=1024,
)

print(response["choices"][0]["message"]["content"])

For long-document use (leveraging the 128K context window):

python

# Load a large document and process it within a single context window
with open("large_document.txt", "r") as f:
    document = f.read()

llm_long = Llama(
    model_path=model_path,
    n_ctx=65536,      # 64K context — adjust based on available VRAM
    n_gpu_layers=-1,
    verbose=False,
)

response = llm_long.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are a document analysis assistant."},
        {"role": "user", "content": f"Summarize the following document and extract all action items:\n\n{document}"}
    ],
    temperature=0.3,
    max_tokens=2048,
)
print(response["choices"][0]["message"]["content"])

For tool-calling with post-tool answer validation (addresses tool_02 final_mismatch finding):

python

import json
from huggingface_hub import hf_hub_download
from llama_cpp import Llama

model_path = hf_hub_download(
    repo_id="pbhappliedsystems/mistral-nemo-instruct-2407-gguf-Q4-K-M",
    filename="mistral-nemo-instruct-2407-gguf-Q4-K-M.gguf"
)

llm = Llama(model_path=model_path, n_ctx=4096, n_gpu_layers=-1, verbose=False)

def execute_tool(tool_name: str, args: dict) -> str:
    """Stub: replace with actual tool execution."""
    if tool_name == "add":
        return str(args["a"] + args["b"])
    raise ValueError(f"Unknown tool: {tool_name}")

def call_with_tool_and_validate(prompt: str) -> dict:
    """
    Scaffolded tool dispatch with post-execution answer validation.
    Addresses quant_eval v7.21 finding: tool_02 final_mismatch at Q4_K_M.
    toolcall stage1 pass rate = 1.000; final answer accuracy is not guaranteed.
    """
    response = llm.create_chat_completion(
        messages=[
            {
                "role": "system",
                "content": "You are a tool-calling assistant. Emit a tool call JSON, then report the result."
            },
            {"role": "user", "content": prompt}
        ],
        temperature=0.0,
        max_tokens=512,
    )
    raw = response["choices"][0]["message"]["content"]

    # Extract tool call
    import re
    match = re.search(r'\{[^{}]*"tool_name"[^{}]*\}', raw, re.DOTALL)
    if not match:
        raise ValueError(f"No tool call found: {raw[:200]}")
    call = json.loads(match.group(0))

    # Execute tool independently — do not trust model's reported result
    actual_result = execute_tool(call["tool_name"], call["args"])
    return {"tool_call": call, "validated_result": actual_result, "model_raw": raw}

result = call_with_tool_and_validate("What is 10 minus 4?")
print(f"Validated result: {result['validated_result']}")

CLI — llama-cli

bash

# One-shot prompt
llama-cli \
  --model mistral-nemo-instruct-2407-gguf-Q4-K-M.gguf \
  --chat-template mistral \
  --system-prompt "You are a precise assistant." \
  --prompt "Analyze the following and return a JSON object with keys: summary, risk_level, action_items." \
  --n-predict 1024 \
  --ctx-size 32768 \
  --n-gpu-layers -1 \
  --temp 0.3

For server deployment (OpenAI-compatible endpoint):

bash

llama-server \
  --model mistral-nemo-instruct-2407-gguf-Q4-K-M.gguf \
  --chat-template mistral \
  --ctx-size 32768 \
  --n-gpu-layers -1 \
  --port 8080 \
  --host 0.0.0.0

Query via the OpenAI-compatible API:

python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-required")

response = client.chat.completions.create(
    model="mistral-nemo-instruct-2407-gguf-Q4-K-M",
    messages=[{"role": "user", "content": "Your prompt here"}],
    temperature=0.3,
)
print(response.choices[0].message.content)

Artifact Provenance

Artifact	Format	Size	SHA256
`mistral-nemo-instruct-2407-gguf-Q4-K-M.gguf`	GGUF Q4_K_M	7.48 GB	`5765024ff3361f6dc5b590b963b378bd2e87ac95eabe5823a08a3ad336b498c9`
F16 (companion repo)	GGUF F16	24.5 GB	`cc7b8c5c3f129ad32aee562017b5e56f1284ee7d92445be292c644a42b3c9556`

Both artifacts were produced from mistralai/Mistral-Nemo-Instruct-2407 using a custom-built llama.cpp conversion and quantization pipeline developed by PBH Applied Systems.

Evaluation Methodology

quant_eval v7.21 is a proprietary behavioral evaluation harness developed by PBH Applied Systems. It evaluates both the full-precision (F16) and quantized variants against an identical fixture set, enabling direct comparison of capability retention across quantization levels.

Fixture set: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c079371b02a94f3c149eb78a6adc03dc16ff6833b964fbf4174f0)

Family	Description	Pass Signals
`fuzz`	Property-based regression; structured placement correctness	schema_ok, constraints_ok
`json`	Single-step structured JSON with constraint rules	schema_ok, constraints_ok
`json_multistep`	Multi-step planning with self-check and oracle verification	schema_ok, checks_consistent_ok, stop_semantics_ok, oracle_equiv_ok
`mcq`	Multiple-choice extraction	choice_ok
`stateful_followup`	Two-turn state tracking; turn-2 correct given turn-1	turn1/2_parse_ok, turn1/2_exact_match
`mixed_brief_json`	Hybrid: natural language answer + valid JSON block	answer_line_ok, json_parse_ok, schema_ok
`toolcall`	Tool call embedded in response; parse + schema validation	stage1_tool_parse_ok, stage1_tool_schema_ok
`toolcall_only`	Bare schema-only tool call; strict tool name + args check	tool_name_ok, args_ok

Evaluation hardware: NVIDIA RTX 4090 (24 GB VRAM) Evaluation date: February 11, 2026 quant_eval seed: 42

🔬 About quant_eval & This Evaluation Series

quant_eval is a proprietary behavioral evaluation harness developed by PBH Applied Systems, LLC. It measures real agent-adjacent task performance across structured output, tool dispatch, multi-turn state retention, and multi-step planning — not perplexity or leaderboard proxies. Every model published under pbhappliedsystems has been independently evaluated using quant_eval before being recommended for any production role.

See it in action: Live AI Agent Demo → The demo runs production-style agent workflows powered by open-weight models selected through the quant_eval evaluation pipeline.

Need a deployment recommendation? Not sure which quantization level is right for your hardware, latency target, or agent type? → pbhappliedsystems.com

Evaluated and published by PBH Applied Systems, LLC · patrick@pbhappliedsystems.com

About PBH Applied Systems

PBH Applied Systems, LLC is an Oklahoma City–based applied machine learning and AI systems company specializing in production-grade model evaluation, quantization pipelines, agentic AI infrastructure, and scalable AI-driven application development. The organization operates with a strong emphasis on engineering rigor, reproducibility, and real-world deployment constraints — particularly in environments where performance, cost efficiency, and reliability must be balanced against available hardware and budget.

Founder — Patrick Hill, M.S.

PBH Applied Systems was founded by Patrick Hill, a Data Scientist and AI/ML Engineer with 10+ years of experience delivering advanced analytics, predictive modeling, and decision-support solutions across high-stakes operational environments. Patrick holds a Master of Science in Software Engineering with concentrations in Artificial Intelligence and Machine Learning and a B.S. in Business Finance.

Technical expertise spans:

Languages & Data: Python, SQL, Linux, Pandas, NumPy, scikit-learn
ML & Modeling: Supervised and unsupervised learning, neural networks, NLP, transformers, regression, classification, forecasting, and feature engineering
AI/ML Frameworks: PyTorch, TensorFlow/Keras, HuggingFace Transformers, GGUF, llama.cpp, BitsAndBytes, PEFT, QLoRA
Deployment & MLOps: Flask APIs, Docker, CI/CD pipelines, REST endpoints, streaming inference, version control
Data Platforms: Jupyter, Databricks, Power BI, Matplotlib
Quantization: GGUF conversion, Q4_K_M / Q5_K_M / Q8_0 strategies, adapter-per-model evaluation architecture

Published Author

Patrick is the author of Applied Machine Learning: Concepts, Tools, and Case Studies — a 1,200+ page practitioner-oriented textbook adopted as required reading for CSC 373 – Machine Learning at the University of Advancing Technology.

Core Service Areas

1. LLM Optimization & Deployment — End-to-end GGUF conversion and quantization with custom llama.cpp pipelines and adapter-per-model architecture.

2. AI Evaluation Frameworks — Proprietary behavioral evaluation via quant_eval: per-family pass rates, F16 vs. quantized delta analysis, failure cluster diagnostics, deployment recommendations.

3. Agentic AI Infrastructure — LlamaIndex ReAct agents, Flask orchestration, serverless GPU inference, full pipeline from model selection to production serving.

4. Scalable AI Application Development — Multimodal applications (quantized LLMs + Whisper + BLIP), Dockerized Flask APIs, advanced time-series forecasting with custom attention mechanisms, Bayesian hyperparameter optimization, and FinBERT sentiment fusion.

5. ML Pipeline Design & Analytics — Feature engineering, forward-chaining cross-validation, KPI dashboards, analytical governance at scale.

6. Model & Agent Cataloging — Structured catalog publishing with reproducible artifacts and clear performance tradeoff documentation.

📞 Work With PBH Applied Systems

The complete breakdown of ms_hard_01 at Q4_K_M — all four gating signals failing simultaneously — and the tool_02 final_mismatch are findings that only appear when you run both the F16 and quantized variant against the same behavioral test suite. Neither shows up in standard benchmarks. Neither is visible from casual testing. Both have direct consequences for production deployment decisions.

A model that dispatches tools correctly but gives wrong answers, and that fails completely on hard planning cases, needs to be known before it goes to production — not after.

👉 Book a Scoping Call — Discuss your model selection, quantization strategy, or deployment architecture directly with Patrick.

👉 Request an Evaluation Report — A full quant_eval behavioral audit: per-family pass rates, F16 vs. quantized delta analysis, failure cluster diagnostics, and a deployment recommendation. Engagements from $2,500.

Connect


🌐 Website	pbhappliedsystems.com
📧 Email	patrick@pbhappliedsystems.com
💼 LinkedIn	PBH Applied Systems, LLC
▶️ YouTube	@pbhappliedsystems
📸 Instagram	@pbhappliedsystems
👍 Facebook	pbhappliedsystems

License

This GGUF repository inherits the license of the base model: Apache 2.0 — mistralai/Mistral-Nemo-Instruct-2407

The quant_eval evaluation methodology, fixture set, and scoring framework are proprietary to PBH Applied Systems, LLC and are not included in this repository.

GGUF conversion, quantization, and behavioral evaluation performed by PBH Applied Systems, LLC · quant_eval v7.21 · Run ID: 20260211_022944

pbhappliedsystems/mistral-nemo-instruct-2407-gguf-Q4-K-M

Mistral-Nemo-Instruct-2407 · GGUF Q4_K_M

Try This Model in the Live AI Agent Demo

Model Description

Key Characteristics

PBH Applied Systems Evaluation — quant_eval v7.21

Aggregate Scores (Q4_K_M)

Per-Family Pass Rates

F16 Baseline (full_weight_transformers)

Q4_K_M (quantized_llama_cpp)

Key Findings

Finding 1: json_multistep — Complete Breakdown on Hard Case

Finding 2: toolcall — Final Mismatch on tool_02 (Q4_K_M)

Finding 3: toolcall_only — Consistent args Failure with Stable Tool Name

Finding 4: MCQ "got=A" Bias

Signal-Level Diagnostics (Q4_K_M)

json_multistep

stateful_followup

toolcall_only (Q4_K_M)

mixed_brief_json

Recommended Use Cases

✅ Deploy with Confidence (Q4_K_M)

⚠️ Use with Guardrails (Q4_K_M)

❌ Not Recommended (Q4_K_M)

128K Context Window — Deployment Considerations

Hardware Requirements

Usage

Installation

Python — llama-cpp-python

CLI — llama-cli

Artifact Provenance

Evaluation Methodology

🔬 About quant_eval & This Evaluation Series

About PBH Applied Systems

Founder — Patrick Hill, M.S.

Published Author

Core Service Areas

📞 Work With PBH Applied Systems

Connect

License

pbhappliedsystems/
mistral-nemo-instruct-2407-gguf-Q4-K-M

F16 Baseline (`full_weight_transformers`)

Q4_K_M (`quantized_llama_cpp`)