You can deploy this model and start earning money today!

Views

No views yet

OculusMind-ToolCall-8B ("Pico")

Name: OculusMind-ToolCall-8B-v1
Availability: InStock

An OculusMind.AI model, fine-tuned from Mistral AI's Ministral-3-8B-Instruct-2512.

"Pico" is this model's own name for itself. Ask it what it is with no system prompt of your own and it will say so: the name is baked into the default system message embedded in the GGUF builds, the way the base model calls itself "Le Chat". It is the same model as OculusMind-ToolCall-8B-v1 — the results tables below use "Pico" purely to keep the columns narrow.

A LoRA fine-tune of mistralai/Ministral-3-8B-Instruct-2512-BF16 for multi-turn tool-calling agent workloads, released under Apache-2.0 by OculusMind.AI.

What this is. A LoRA fine-tune aimed at tool calling. It improves single-turn function calling substantially over its base, does not improve multi-turn tool use, and becomes less willing to decline a call than the base — both directions are in §3, in the same tables. It did not meet our own internal promotion rule; §7 gives that in full. Successor versions are in training.

This card exists so that a business evaluator, or their engineer, can check our work. Every number below is either (a) measured by us on a pinned public benchmark that the reader can rerun, or (b) a count from a private suite, labelled as such, with the denominator shown. Nothing is quoted from a third-party leaderboard as if it were our measurement, and nothing is cherry-picked: the full per-category benchmark table is published, including every category where the fine-tune is worse than its base.

Run it: USAGE-OLLAMA.md — verified end to end on Ollama 0.32.5 against the files in this repo, both quantizations, including structured tool calls and the tool-result round trip.
Weights: Apache-2.0. See LICENSE.
Base: mistralai/Ministral-3-8B-Instruct-2512-BF16 (Apache-2.0), modified. See NOTICE.
Training data: all external sources Apache-2.0; 53 rows are OculusMind internal production captures, which remain unpublished and proprietary. The captures are not redistributed; the weights are. See §6.
Status of this card: every figure below describes a measured artifact in this repository — nothing is projected, estimated or carried over from an earlier build. pipeline/publish/preflight.py re-verifies the card against the scored results and the shipped weights before staging, and refuses on any mismatch.

1. Summary


Base model	`mistralai/Ministral-3-8B-Instruct-2512-BF16`
Base revision (pinned)	`f6fae9795746f63c9be8344932f01275f3c63734`
Base parameters	8.92 B (8,918.0 M)
Base architecture	`mistral3` (`Mistral3ForConditionalGeneration`)
Method	LoRA (rank 16), supervised fine-tuning, merged into the base weights
Trainable parameters	44,564,480 (0.50% of base), all 34 language-model layers
Training rows	14,456 consumed from a 14,465-row mix (see `training/DATA.md`)
Optimizer updates	1,807 (1 epoch, batch 1, gradient accumulation 8)
Seeds trained	1 (seed 42). Seeds 43/44 cancelled — see §7
Context length	262,144 native (YaRN, base 16,384); we serve and evaluate at 65,536
Released formats	GGUF `Q4_K_M` and `Q8_0`; LoRA adapter. No merged `safetensors` — see `UPLOAD-MANIFEST.md`
Headline instrument	BFCL V4, harness pinned at `f7cf7359b7ac615a0b294831c5ba2bc95ee4a000` (`bfcl-eval` 2025.12.17)
Headline result	`Q4_K_M`: Non-Live AST +14.25, Live AST +12.88. `Q8_0`: +11.62 / +11.03. Each quantization is compared against a base built at the same quantization
Multi-turn	no gain at either quantization — `Q4_K_M` +1.00, `Q8_0` −1.00, every category inside its band
Countervailing result	`live_irrelevance` −16.18, `irrelevance` −7.08; private suite 0.839 → 0.726
Internal decision	Not promoted — it did not meet our internal bar and is not deployed in any OculusMind.AI product. Reasons in §7

The claim, precisely stated

On BFCL V4, measured by us on one machine with a pinned harness commit, comparing this model's Q4_K_M build against the base checkpoint's Q4_K_M build across 4,441 judge-free items, this model improves single-turn function calling — Non-Live AST 73.71 → 87.96, Live AST 66.25 → 79.13.

Multi-turn does not improve. All four multi-turn categories land inside their 95 % noise bands (16.38 → 17.38 in aggregate). Two restraint categories regress beyond their bands — live_irrelevance 85.29 → 69.12 on 884 items, and irrelevance 89.58 → 82.50 — and our private production agent-workload suite regresses from 52/62 to 45/62. The result was not replicated across seeds; one seed was trained.

What this claim is not

It is not a claim that this model is good. It is a claim that a specific, reproducible measurement moved a specific amount in each direction.
It is not a claim of parity with, or superiority to, any frontier model.
It is not a BFCL leaderboard submission. We did not submit anything to Berkeley; there is no third party vouching for these numbers. That is exactly why the harness commit, the exact commands, and our own base-checkpoint measurement are all published here — so you can rerun it.
It is not a general capability claim. This model was trained and measured on tool-calling and agent trajectories. See §8 Limitations.
It is not replicated. Our own promotion rule requires three seeds and got one. A single-seed result cannot distinguish a recipe effect from a lucky draw, and we do not ask you to treat it as if it could.

2. What changed relative to the base

We applied LoRA supervised fine-tuning aimed at this base checkpoint's measured deficits. Those deficits were measured first — but, importantly for reading the result, they were measured on our internal suite only:

Deficit (base, measured by us)	Instrument	Base	Reference
Multi-step trajectory execution	Private suite, `multi-step` family (3 cases)	1/3	Claude Haiku 4.5, our measurement: 3/3
Tool selection	Private suite, `tool-use` family (23 cases)	18/23	Claude Haiku 4.5, our measurement: 23/23
Clarification (ask, don't invent)	no evidence available at build time	—	—
Irrelevance detection	no evidence available at build time	—	—

The resulting mix weights were multi-turn trajectory 0.5545, single-turn selection 0.2623, irrelevance negatives 0.1000, clarification 0.0832.

This is the most important methodological fact on this page. The two buckets with no measured evidence behind them were clarification and irrelevance — their weights came from a prior, not from a measurement, with the irrelevance weight clamped to a 10% floor by a guardrail. Irrelevance is precisely what regressed. We did not have a BFCL profile of the base when the mix was built, so we could not see that the base was already strong at irrelevance (89.58 / 85.29) and had nothing to gain and much to lose there. Weighting a bucket by prior instead of by measurement is how a training recipe walks into a regression it could have predicted.

Composition, licences and per-source counts: training/DATA.md. Hyperparameters, seed and hardware: training/RECIPE.md.

Provenance note, stated plainly. Published Berkeley BFCL V4 leaderboard values for Claude-Haiku-4-5-20251001 (FC) were recorded in this build's manifest as background context, but no bucket weight was derived from them — every weight in this mix traces to our own private-suite measurement or to a prior. Those published values are not our measurements and are not used to support any claim about this model. The one Haiku number we report as evidence — the private-suite count in §4 — is our own measurement and is labelled as such. See eval/METHODOLOGY.md §5 for why the two are kept strictly separate.

3. Primary evidence — BFCL V4

Berkeley Function Calling Leaderboard V4, harness pinned at gorilla commit f7cf7359b7ac615a0b294831c5ba2bc95ee4a000 (bfcl-eval==2025.12.17), 17 categories, 4,441 items, all scored offline (AST scoring and local mock backends; no network calls, no API keys, no LLM judge).

Both rows below are our own measurements on the same harness commit, the same machine, and the same serving stack. The base row is not quoted from Mistral and not quoted from the leaderboard. Mistral publishes no function-calling benchmark for Ministral 3, and the model has no Berkeley leaderboard entry, so the base row here appears to be the first public BFCL figure for this checkpoint.

Both rows are Q4_K_M: the base was quantized from its own pristine weights by the same converter at the same setting, so quantization is held constant and the delta is attributable to the fine-tune rather than to the build.

The verdict column is computed, not editorial: a delta is called noise unless it exceeds the 95 % band for the difference of the two rates. Tables in this section are generated by pipeline/bench/card_tables.py directly from the scored summary.json, never transcribed.

Single-turn AST — non-live

category	items	base Q4_K_M	Pico Q4_K_M	delta	95 % band	verdict
`simple_python`	400	88.50	95.00	+6.50	±3.79	gain
`simple_java`	100	20.00	62.00	+42.00	±12.33	gain
`simple_javascript`	50	32.00	66.00	+34.00	±18.43	gain
`multiple`	200	88.50	97.00	+8.50	±5.01	gain
`parallel`	200	83.50	92.00	+8.50	±6.37	gain
`parallel_multiple`	200	76.00	88.50	+12.50	±7.39	gain

Single-turn AST — live

category	items	base Q4_K_M	Pico Q4_K_M	delta	95 % band	verdict
`live_simple`	258	59.69	82.95	+23.26	±7.54	gain
`live_multiple`	1053	68.19	78.63	+10.45	±3.75	gain
`live_parallel`	16	50.00	56.25	+6.25	±34.51	noise
`live_parallel_multiple`	24	62.50	75.00	+12.50	±25.99	noise

Multi-turn — no category clears its band

category	items	base Q4_K_M	Pico Q4_K_M	delta	95 % band	verdict
`multi_turn_base`	200	23.50	23.00	−0.50	±8.28	noise
`multi_turn_miss_func`	200	9.50	7.50	−2.00	±5.46	noise
`multi_turn_miss_param`	200	14.00	19.00	+5.00	±7.26	noise
`multi_turn_long_context`	200	18.50	20.00	+1.50	±7.73	noise

Restraint — two categories regress beyond their bands

category	items	base Q4_K_M	Pico Q4_K_M	delta	95 % band	verdict
`irrelevance`	240	89.58	82.50	−7.08	±6.17	regression
`live_irrelevance`	884	85.29	69.12	−16.18	±3.84	regression
`live_relevance`	16	81.25	87.50	+6.25	±25.07	noise

Eight categories improve beyond their band, two regress beyond it, and seven move inside it — including all four multi-turn categories.

Aggregates (computed with the leaderboard's own definitions, through the single aggregation path in pipeline/bench/bfcl_aggregate.py — see eval/bfcl-results.md for the arithmetic):

Block	Base Q4_K_M	Pico Q4_K_M	Δ
Non-Live AST	73.71	87.96	+14.25
Live AST	66.25	79.13	+12.88
Multi-Turn	16.38	17.38	+1.00 (no gain)

And at Q8_0, each side again built at the same quantization — full table in eval/bfcl-results-q8_0.md:

Block	Base Q8_0	Pico Q8_0	Δ
Non-Live AST	76.31	87.94	+11.62
Live AST	67.95	78.98	+11.03
Multi-Turn	17.38	16.38	−1.00 (no gain)

The Q8_0 comparison is the less flattering one, and it is the more faithful one. A higher-fidelity base scores better (76.31 against 73.71 Non-Live), so the same fine-tune buys less: +11.62 rather than +14.25. Multi-turn turns slightly negative. Category tallies move the same way — 8 gains and 2 regressions at Q4_K_M, 7 gains and 3 regressions at Q8_0.

Why the two differ, precisely. It is not that this model does better at 4-bit. Counting correct items out of 4,441:

	`Q4_K_M`	`Q8_0`	shift
Base	2,894	2,942	+48 items
Pico	3,061	3,053	−8 items

Pico is essentially insensitive to the quantization — 0.18 % apart, well inside run-to-run variation (sampling is at 0.001, not greedy). The base is not: it gains 48 items at 8-bit, consistently, in 16 of 17 categories. So the Q4_K_M delta is larger because its baseline is depressed, not because the fine-tune is better there.

We do not know why the fine-tune is the more quantization-robust of the two. The obvious guess — that training sharpened its tool-call output distribution, so 4-bit rounding flips its chosen token less often — is a hypothesis we have not measured, and we are not asserting it.

If you want one number from this card, take the Q8_0 one: it is the comparison where quantization damage is smallest on both sides.

Deltas are the difference of the published two-decimal figures.

3.2 The categories BFCL leaves out of those aggregates, and which got worse

BFCL's three headline blocks exclude the relevance/irrelevance detection categories. A model can therefore post a large headline gain while getting materially worse at knowing when not to call a tool, and the headline will not show it. Ours did — which is why those categories are in the main table above rather than in an appendix.

Read together: the model became more willing to emit a tool call. Where a call is warranted that is a large win (live_simple +23.26, simple_java +42.00, simple_javascript +34.00). Where restraint is correct it is a real loss (irrelevance −7.08, live_irrelevance −16.18). live_relevance — the category that rewards calling when a call is appropriate — moved up, the same effect seen from the other side.

live_irrelevance is the single largest regression in the whole sweep, on 884 items. If your workload sends the model requests it should decline — open-ended user chat, requests naming tools that do not fit, anything where a wrong call is costlier than no call — the base model is better at that than this one, and you should use the base.

multi_turn_miss_func (−2.00, inside the band) is the same story from a third side: that category rewards asking for a function the model has not been given rather than reaching for one it has. It did not improve either.

What we deliberately do not report. No BFCL "Overall" score. Overall at this harness pin weights agentic web-search and memory blocks that we exclude (they require live HTTP and third-party search API keys, which we do not run). An Overall computed over a subset of categories is not comparable to a published Overall, so publishing one would be misleading. The excluded categories are listed in eval/METHODOLOGY.md §2.

Replication: none. Our promotion rule requires three seeds. One was trained. See §7.

Reproduce it yourself: eval/reproduce.md.

4. Secondary evidence — private agent-workload suite (counts only)

In our release process this section is normally a non-regression guard — the check that a benchmark gain did not cost real production behaviour. On this model it is the countervailing result, and it is the reason the model was rolled back.

It is an OculusMind-internal suite of recorded production agent workloads. The suite itself, its cases, its prompts, its rubrics and its tool schemas are not published — they encode our production system. What we publish is counts with denominators, and a content-free description of how scoring works (eval/METHODOLOGY.md §3).

Suite version v2, 67 cases, 5 cases excluded as environment artefacts → 62 scored cases. Case-level pass requires every criterion on the case to pass.

Capability family	Cases	Base	This model	Δ	Claude Haiku 4.5 †
tool-use	23	18	16	−2	23
memory	16	14	12	−2	16
safety	9	8	7	−1	9
general	6	6	5	−1	6
grounding	4	4	4	0	4
multi-step	3	1	1	0	3
formatting	1	1	0	−1	1
Total	62	52 (0.839)	45 (0.726)	−7	62 (1.000)

Holdout subset (24 artefact-free cases): base 20 → this model 15.

Read as paired flips, not as two independent proportions. Case-by-case, 9 cases went pass → fail and 2 went fail → pass. On a 62-case denominator, "0.839 versus 0.726" sits close to overlapping margins; "9 broke, 2 fixed" does not. The erosion is spread across five of the seven families, which is what makes it a behavioural shift rather than a single flaky case.

Verdict: FAIL. This is a regression on the workloads the model exists to serve. It was not promoted, on 2026-07-30, and the model was not promoted.

Full statement, the raw 67-case figures, and the flip list by family: eval/internal-results.md.

† This Haiku figure is our own measurement, not a vendor claim and not an endorsement. We ran claude-haiku-4-5-20251001 against our own private suite, scored by our own judge, under the same protocol as the other two columns. Anthropic did not produce, review or approve this number, and it should not be read as a public benchmark result. It is here as a reference ceiling for our own workloads only. See eval/internal-results.md for the full statement.

Why 5 cases are excluded. Our capture harness ran one document-storage capability against a local backend that reports the capability as unavailable, whereas the production system always has it available. Five cases therefore recorded — and then graded — a scenario that cannot occur in production. We found this after those cases had already been scored, and disclosing it costs us rather than helps us: all five of the Haiku reference model's failures were those cases, so excluding them moves the reference from 0.925 to a perfect 1.000, i.e. it raises the bar we are measured against. It also makes this model's regression look slightly smaller than the raw figures would (0.791 → 0.672 raw, versus 0.839 → 0.726 artefact-free), so we publish both. Full statement: eval/internal-results.md §3.

5. Decontamination

Training-set contamination is the standard and correct reason to dismiss a fine-tune's benchmark gains — and this model's benchmark gains are large enough that the question deserves a real answer. We screened every external source against the pinned BFCL corpus before any row was used, and we publish the reports.

Method: 8-gram overlap on task content (not whole rows — shared prompt boilerplate is not contamination), plus function-signature matching, against a forbidden corpus built from the pinned BFCL data directory (10,109 benchmark items, 194,092 n-grams, 2,467 function signatures) and our private suite items. Gate: a source fails if more than 2% of benchmark items are covered by it.

Source	Rows scanned	BFCL items hit	Hit rate	Gate (≤2%)
ToolACE	13,301	137 / 10,109	1.36%	PASS
hermes-function-calling-v1	26,274	95 / 10,109	0.94%	PASS
hermes_reasoning_tool_use	113,259	142 / 10,109	1.41%	PASS
smol-smoltalk	54,608	126 / 10,109	1.25%	PASS

An inline re-screen at dataset build time dropped a further 643 rows (642 on signature matching, 1 on coverage). Full method, per-source report hashes and the source file SHA-256s: eval/decontamination.md.

We never train on the BFCL corpus itself, in whole or in part. It is registered in our source list as NEVER TRAIN.

What this does not settle. Decontamination screens our data, not the base model's pretraining corpus. Both sides of our delta carry whatever the base carries, so it cancels out of the delta and not out of the absolute numbers. See eval/decontamination.md §6.

6. Training data

14,465 rows built; 14,456 consumed (4 over-length rows dropped whole at load, 5 rows held out for validation). Every external source is Apache-2.0; none is share-alike; none is non-commercial.

Source	Licence	Rows in the mix
`Team-ACE/ToolACE`	apache-2.0	4,650
`interstellarninja/hermes_reasoning_tool_use`	apache-2.0	3,838
`NousResearch/hermes-function-calling-v1`	apache-2.0	3,031
`HuggingFaceTB/smol-smoltalk` (general-chat replay)	apache-2.0	2,893
OculusMind internal capture (not published)	proprietary, ours	53
Total		14,465

Counts are read from the shipped dataset's own manifest and re-counted from the shipped train.jsonl/valid.jsonl; both agree.

MadeAgents/xlam-irrelevance-7.5k (CC-BY-4.0) was pulled and screened but contributed zero rows — all 7,500 were excluded upstream by a synthesis gate before selection. Its attribution obligation therefore does not attach to this model, and it is now excluded from our commercial mixes by policy.

The internal anchor is 53 rows against 14,412 external rows — a ratio of roughly 328:1. That ratio is the leading diagnosis for the private-suite regression, and it is stated here rather than buried in the appendix. See §7.

Full composition, per-source screening funnel, and the licence verification dates: training/DATA.md.

On what is and is not redistributed. The weights are Apache-2.0. The 53-row internal capture slice is not published and is not redistributable — it is recorded production traffic from OculusMind's own agent stack. An open licence on the weights does not open the data, and this card does not claim to open it.

7. The promotion rule, and why this model did not meet it

Five earlier fine-tuning iterations in this programme were not promoted before this one. We publish that record because it is the context for reading this model honestly.

The promotion rule applied to this model was fixed before it was trained:

≥1 BFCL category improves beyond its band, and
no BFCL category regresses beyond its band, and
(1) and (2) replicate across three training seeds, and
the private-suite count does not regress against the base.

Result of applying the rule: FAIL, on three of the four conditions.

#	Condition	Outcome	The numbers that decided it
1	≥1 category clears its band	PASS	8 categories improve beyond their band; the largest is `simple_java` +42.00 (band ±12.33)
2	no category regresses beyond its band	FAIL	`live_irrelevance` −16.18 (band ±3.84) and `irrelevance` −7.08 (band ±6.17)
3	replicated across three seeds	FAIL	one seed trained
4	private suite does not regress	FAIL	52/62 → 45/62; 9 cases broke, 2 were fixed

On the missing seeds, plainly. Seeds 43 and 44 were not trained. Our replication rule requires the worst seed to hold the family floors; seed 42 broke them by a wide margin on condition 4, so no result from seeds 43 or 44 could have produced a promotion. Under our early-termination rule, once a completed seed mathematically determines the verdict the remaining seeds are killed and the compute is redirected to diagnostics. That is what happened here — approximately 32 machine-hours went to the BFCL diagnostic that produced §3 instead of to refining a foregone conclusion. This is an honest saving of compute and it is also a real limitation of the published numbers: nothing in §3 has been shown to replicate.

Why we are publishing it anyway

A model that fails an internal promotion gate is normally deleted. Three reasons this one is published instead:

The single-turn gain is real, large and independently checkable. +14.25 Non-Live AST and +12.88 Live AST, on 4,441 judge-free items with a pinned harness anyone can rerun, against a base we quantized and measured the same way. simple_java moves 20.00 → 62.00 and live_simple 59.69 → 82.95. The multi-turn gain we once claimed is not among these reasons — it did is not among them: multi-turn does not improve (§3).
The failure is the more useful finding. A recipe that produces a large public-benchmark gain and a production regression is a thing practitioners should be able to read about, with both numbers in the same table.
The base row has value on its own. Mistral publishes no function-calling benchmark for Ministral 3 and it has no Berkeley leaderboard entry, so the base measurement in §3 appears to be the first public BFCL figure for this checkpoint.

It is published as a research release, not as a product, and the status block at the top of this card says so before any number does.

The diagnosis

The recipe teaches the model to call tools more. That single mechanism explains the entire result set:

Where a call is warranted, it helps enormously — single-turn AST, across every one of the six non-live categories and the two large live ones.
Where restraint is the correct behaviour, it hurts — irrelevance detection, and asking for a withheld function instead of reaching for a present one.
On our production workloads, where a large fraction of correct behaviour is not calling a tool, or calling a specific domain tool rather than a generic one, it is net negative.

The corollary matters for anyone considering this recipe: its single-turn benefit is inversely proportional to how strong the base already is, and its multi-turn benefit is zero regardless. We ran the same recipe on a second, stronger base (Qwen3.5-4B) and measured both, all figures through the same pinned harness and the same aggregation code as §3:

	Ministral-3-8B (this model)	Qwen3.5-4B
Base Non-Live AST	73.71	82.83
Trained Non-Live AST	87.96 (+14.25)	87.96 (+5.13)
Base Multi-Turn	16.38	37.12
Trained Multi-Turn	17.38 (no gain)	29.25 (−7.87)
Base `irrelevance`	89.58	86.25
Trained `irrelevance`	82.50 (−7.08)	46.25 (−40.00)

Read it honestly and it says three things. Single-turn headroom is real and it is spent: the weaker base gained 14.25 points, the stronger one 5.13, and both land at 87.96. Multi-turn is not what this recipe buys — the weak base did not improve and the strong base got materially worse, so an earlier version of this section, which credited a large multi-turn gain here, was wrong on both its evidence and its conclusion. Restraint is what you pay with, and the stronger base paid catastrophically: 40 points of irrelevance detection.

If your base already handles tool calls well, this recipe has little to give you and a great deal to take.

The Qwen3.5-4B column is our own measurement of a model we have not released, so you cannot reproduce it from anything in this repository. It is published because withholding the one comparison that constrains the claim would be worse than publishing an unverifiable one. Both Qwen runs used the same harness pin, the same serving wrapper and the same Q4_K_M quantization, and were measured twelve minutes apart in a single session — a tighter pairing than this model's own base comparison. No claim about this model rests on it; those stand on the reproducible §3 table alone.

A second contributing factor is dilution: 53 internal anchor rows against 14,412 external rows means the production-specific behaviours received approximately 0.30% of the gradient — 44 internal rows against 14,412 external, since of the 53 internal rows 5 were held out for validation and 4 were dropped at load, so the optimizer never saw 53. Both mechanisms point the same way and we do not claim to have separated their contributions.

The gate stack behind these decisions, and the keep/discard rule itself, is documented in training/METHODOLOGY.md.

8. Limitations and what we did not test

Stated in full, because an evaluator's time is better spent on the gaps than on rediscovering them.

The headline limitation

This model is worse than its base at declining to call tools, and worse than its base on our production agent workloads. If you are deploying an agent, that is likely to matter more to you than the AST gains. Do not adopt this model on the strength of §3 without reading §4.

Benchmark scope

We ran 17 of BFCL V4's categories. We did not run the agentic web-search categories (web_search_base, web_search_no_snippet) — they require live HTTP and a third-party search API key. We did not run the memory categories (memory_kv, memory_vector, memory_rec_sum) or format_sensitivity. No BFCL "Overall" score is reported for that reason.
All BFCL runs were in function-calling (FC) mode, not prompt mode. Prompt mode is untested.
We ran no other public benchmark. No τ²-bench, no AgentBench, no ToolBench, no API-Bank, and no general-knowledge, reasoning, math, coding, long-context or multilingual benchmark. We make no claim about any of them, and a fine-tune targeted this narrowly can plausibly cost general capability. We measured the specific regression risk we could measure (BFCL categories + private suite) and not the rest.
The base is a multilingual model (11 declared languages). All of our evaluation is English. Our training data is predominantly English with one exception worth naming: Team-ACE/ToolACE — 4,650 rows, 32 % of the mix — is tagged en and zh upstream, so some Chinese content entered training. We did not evaluate in Chinese or any other language, so multilingual behaviour after fine-tuning is entirely unmeasured, and the presence of that data is not a claim that it helped.

Statistical scope

One seed. This is the largest statistical limitation on the page. Three seeds is our own standard and it was not met. A single-seed result cannot separate a recipe effect from a draw.
BFCL category sizes vary from 16 to 1,053 items. On the smallest categories (live_parallel n=16, live_relevance n=16, live_parallel_multiple n=24) the band exceeds ±25 points and no delta there is meaningful. live_parallel +6.25 sits inside a ±34.51 band — read it as noise. Sixteen items constrain very little.
Standard errors are binomial and assume item independence. BFCL multi-turn items within a trajectory are not fully independent, so multi-turn intervals are, if anything, optimistic.

Private-suite scope

62 scored cases is a small denominator. A one-case change is ~1.6 points. The −7-case movement reported in §4 is well outside that, but do not read the per-family sub-counts (some as small as 1 case) as individually meaningful.
The suite measures OculusMind's production agent workloads. It is not a general agent benchmark, and a weak score on it does not mean the model is weak on yours — only that it is weak on ours.
Scoring uses deterministic tool-call gates plus a rubric judge. The judge is a local open-weights model calibrated against an independent reference judge (threshold agreement 0.9191 over 383 samples). A calibrated judge is still a judge; it is not an oracle.
The suite is not published, so this column is not independently verifiable. That is precisely why it is the secondary claim and BFCL is the headline — and note that in this case the unverifiable instrument is the one reporting the bad news, which is the direction that costs us.

Modality and deployment scope

The base checkpoint is multimodal (Mistral3ForConditionalGeneration, with a 24-layer vision tower). The LoRA adapter touched only language_model.* tensors — the vision tower received no gradient and is unchanged. Note this is a property of the adapter, not of any merged artifact: the merge drops the vision tower entirely, so no released weight file contains it.
The released GGUF is text-only: the conversion emits 309 tensors, which is the language model alone. Any image path in that serving artefact does not exist. This release ships no merged safetensors at all, so the question does not arise for a weights file: the two GGUF builds are text-only (the merge emits 309 of the base's 531 tensors and all 222 vision and projector weights are dropped). The LoRA adapter is the one artifact here that preserves the base's vision capability — it touches text-tower projections only, so applying it to the unmodified base yields a multimodal model. See UPLOAD-MANIFEST.md.
Measured on Apple Silicon with a llama.cpp GGUF Q4_K_M serving path and a 65,536-token serving window, against an OpenAI-compatible endpoint. Behaviour at other quantisations, on other runtimes (vLLM, SGLang, TensorRT-LLM), or at the base's full 262,144-token context is unmeasured.
Serving-path caveat. Mistral's supported path is mistral-common plus vLLM's native Mistral tool parser. We ran our own OpenAI-compatible wrapper; its template is cross-validated against mistral-common with no differences, but multi-turn tool rendering on this family is a known integration hazard, and a different client may produce different numbers.
Throughput: measured back-to-back, and it is a non-event. All four builds in one session on an idle M3 Ultra, identical prompt and settings, medians of three runs of 128 generated tokens:

build size prompt generation
base Q4_K_M 4.84 GiB 1274.9 tok/s 88.6 tok/s
Pico Q4_K_M 4.84 GiB 1284.4 tok/s 89.0 tok/s
base Q8_0 8.41 GiB 1331.5 tok/s 63.3 tok/s
Pico Q8_0 8.41 GiB 1339.3 tok/s 63.5 tok/s

The fine-tune costs nothing in speed. Within each quantization the two models are within 0.5 % of each other, which is measurement noise — as it should be, since a merged LoRA changes weight values, not architecture or tensor count. Choose the quantization on quality and memory, not throughput: Q8_0 is ~29 % slower to generate and 74 % larger.

Measured 2026-08-19 in a single session on an idle machine; figures from runs taken on different days or a loaded host are not comparable to these.

Safety scope

We ran no dedicated safety, red-team, jailbreak, bias or toxicity evaluation. The private suite contains 9 cases in a safety family drawn from our own production workloads; that is a workload regression check, not a safety evaluation — and this model lost one of those 9 cases relative to its base. We do not know whether that generalises, because we did not run a safety evaluation that could tell us.
The base model's own safety properties are Mistral's, and we have not verified that fine-tuning preserved them. A fine-tune that measurably reduced the model's willingness to decline an irrelevant request is a fine-tune whose willingness to decline other things deserves testing we did not do.
Do not deploy this model in a safety-critical path without your own evaluation.

Known base-checkpoint weaknesses that we did not close

multi_turn_miss_func was the base's single weakest BFCL category at 9.50 and the training did not move it (7.50, −2.00, inside the band). The behaviour it measures — asking for a function you have not been given, rather than misusing one you have — is unchanged. More broadly, no multi-turn category moved: see §3.
On the private suite, the base's multi-step family was 1/3 and remains 1/3.
The base's 10 private-suite failures were concentrated in tool-use (5) and memory (2); after fine-tuning the failure count in those families rose to 7 and 4 respectively.

9. Intended use

Intended. Research and evaluation. Studying the effect of tool-calling supervised fine-tuning on a base with weak measured tool-use. Reproducing or challenging the measurements in §3. Evaluation of OculusMind.AI's model-development methodology by prospective business partners.

Not intended. Production agent deployments; any workload where calling a tool when none is warranted is costly; unsupervised operation in high-stakes domains (medical, legal, financial advice, safety-critical control); any use where the operator has not read §4 and §8.

The weights are Apache-2.0 and you may use them accordingly — the restrictions above are engineering advice about a model we know to be flawed, not licence terms. If this section and LICENSE ever appear to conflict, the licence governs.

10. Licence and attribution

This model's weights are released under the Apache License, Version 2.0. See LICENSE. You may use, modify and redistribute them, including commercially, subject to that licence's conditions.

Base model. Derived from mistralai/Ministral-3-8B-Instruct-2512-BF16, © 2025 Mistral AI, licensed under the Apache License, Version 2.0. Apache-2.0 requires that the original copyright notice and licence text be retained and that modifications be stated. Both obligations are met in NOTICE, which records the base repository, the pinned revision, the retained licence, and an explicit statement of what we changed. Licence re-verified live against the Hugging Face repository metadata on 2026-08-02.

Training data. All external sources are Apache-2.0 (see §6 and training/DATA.md). No CC-BY, CC-BY-SA or non-commercial source contributed a row to this model. The 53-row OculusMind internal capture slice is proprietary and is not published or redistributed; the Apache-2.0 grant covers the released weights, not that data.

Third-party marks. "Claude" and "Haiku" are marks of Anthropic PBC; "Mistral", "Mistral AI" and "Ministral" are marks of Mistral AI; "Berkeley Function Calling Leaderboard" / "BFCL" and "Gorilla" are projects of the University of California, Berkeley; "Hugging Face" is a mark of Hugging Face, Inc. Their appearance here is descriptive reference to measurements and provenance and does not imply any affiliation, sponsorship or endorsement.

11. Files in this repository

Path	What it is
`README.md`	this card
`LICENSE`	Apache License 2.0, covering the released weights
`NOTICE`	Apache-2.0 attribution for the base + statement of modifications
`eval/METHODOLOGY.md`	how both instruments work; the content-free suite description
`eval/bfcl-results.md`	full per-category BFCL table with aggregate arithmetic
`eval/internal-results.md`	private-suite counts, the regression, the quarantine, the Haiku labelling statement
`eval/decontamination.md`	method, thresholds, per-source reports, file hashes
`eval/reproduce.md`	exact commands to re-measure the BFCL result
`eval/models-evaluated-and-not-selected.md`	the roster this base came from, and where it sat in it
`training/DATA.md`	source composition, licences, screening funnel
`training/RECIPE.md`	LoRA config, hyperparameters, seed, hardware, wall-clock
`training/METHODOLOGY.md`	the gate stack and the promotion rule that blocked this model
`CHANGELOG.md`	version history

12. Contact

Andrew Paris LLC
5900 Balcones Dr Suite 21007
Austin, TX 78731
United States

Email: Help@OculusMind.AI

OculusMind.AI is a product name of Andrew Paris LLC. Model repositories are published under the organisation account https://huggingface.co/OculusMindAI.

Citation:

bibtex

@misc{oculusmind_ministral3_8b_agent_tools,
  title  = {OculusMind-ToolCall-8B-v1: a tool-calling fine-tune of Ministral-3-8B-Instruct-2512},
  author = {Andrew Paris LLC (OculusMind.AI)},
  year   = {2026},
  note   = {Apache-2.0 fine-tune of mistralai/Ministral-3-8B-Instruct-2512-BF16
            (Apache-2.0), modified. Regresses on tool-use
            restraint relative to its base.},
  url    = {https://huggingface.co/OculusMindAI/OculusMind-ToolCall-8B-v1}
}

build	size	prompt	generation
base `Q4_K_M`	4.84 GiB	1274.9 tok/s	88.6 tok/s
Pico `Q4_K_M`	4.84 GiB	1284.4 tok/s	89.0 tok/s
base `Q8_0`	8.41 GiB	1331.5 tok/s	63.3 tok/s
Pico `Q8_0`	8.41 GiB	1339.3 tok/s	63.5 tok/s