veyra-tuned-v2b (q4f16_1, MLC)
QLoRA fine-tune of Qwen3-4B-Instruct-2507 for the Veyra process-authoring
task: generating structured process definitions (states, transitions, guards,
tasks, forms) from natural-language intent.
This repo holds the MLC/WebLLM q4f16_1 shards for in-browser inference via
WebGPU. The GGUF (llama.cpp) and LoRA adapter live on the training host.
What it does
Input: a natural-language description of a business workflow.
Output: a JSON object conforming to the Veyra process-definition schema,
including transition.guard_ast (structured predicate objects, not raw DSL
strings).
The model is specialized to the Veyra definition shape: lower_snake_case
keys, declare-before-reference, registered unit catalog keys, exactly one
initial state, terminal states for outcomes, and structured guards.
Training
| |
|---|
| Base | Qwen/Qwen3-4B-Instruct-2507 |
| Method | QLoRA (4-bit NF4, double quant) |
| LoRA rank | 64 (alpha 128, dropout 0.05) |
| Target modules | q, k, v, o, gate, up, down proj |
| Epochs | 3 |
| LR | 2e-4 cosine, 5% warmup |
| Effective batch | 32 (per-device 1 x grad-accum 32) |
| Max length | 4096 |
| Loss masking | response-only |
| Dataset | 709 examples (repair triples + generation gold + distilled), guard-AST distribution |
Trained from base (not continued from v1). A continue-from-v1 attempt (v2a)
failed — the model landed between representations (6.1% clean). Full analysis
in the Step 8 results.
Qualification (114-intent holdout)
| Metric | Target | v2b |
|---|
| Pass-1 parsed | >=90% | 97.4% |
| Pass-1 clean | >=70% | 75.4% |
| Converged valid | >=80% | 93.0% |
| guard_invalid | <=5 | 5 |
Axis M (modify existing workflow, minimal-change fidelity): 93.3%.
Usage (WebLLM / browser)
Variant-reuses the mlc-ai/Qwen3-4B-q4f16_1-MLC WebGPU wasm (same
architecture; no custom wasm compile needed).
1import * as webllm from "https://esm.run/@mlc-ai/web-llm";
2
3const engine = await webllm.CreateMLCEngine("Veyra-Qwen3-4B-v2b-q4f16_1-MLC", {
4 appConfig: {
5 model_list: [{
6 model: "https://huggingface.co/cawalch/veyra-tuned-v2b-q4f16_1-MLC/resolve/main/",
7 model_id: "Veyra-Qwen3-4B-v2b-q4f16_1-MLC",
8 model_lib: "https://raw.githubusercontent.com/mlc-ai/binary-mlc-llm-libs/main/web-llm-models/v0_2_84/base/Qwen3-4B-q4f16_1_cs1k-webgpu.wasm",
9 vram_required_MB: 3431.59,
10 overrides: { context_window_size: 4096 },
11 }],
12 },
13});
14
15const response = await engine.chat.completions.create({
16 messages: [{ role: "user", content: "Request paid time off; manager approves." }],
17 temperature: 0,
18});
Download: 2.2 GB one-time (IndexedDB cached after). Measured 14.9 tok/s
decode on a discrete-GPU MacBook (Chrome, WebGPU). No response_format
constraint — the GBNF grammar engine suppresses guard_ast emission;
Parse+Validate+Repair on the server side is the safety net.
Usage (llama.cpp / server)
The GGUF Q4_K_M (2.5 GB) is served via llama-server. See the Veyra repo
(llm/MODEL_HOST.md) for the serving config.
Limitations
- Domain-specific: optimized for Veyra process definitions, not general chat.
- No
response_format / JSON schema constraint at inference (suppresses
guard_ast; see note above). Output is validated post-hoc.
- Conditional routing (intake-bypass auto-approve guards) is emitted on ~60%
of cases unconstrained; the model has a prior toward unconditional
human-approval chains.
- Context window capped at 4096 for browser VRAM (native 262144).
Files
| File | Purpose |
|---|
params_shard_*.bin (74) | quantized weights (q4f16_1) |
mlc-chat-config.json | MLC chat config (model_id, conv_template: qwen3) |
ndarray-cache.json | weight tensor manifest (symlink to tensor-cache.json) |
tensor-cache.json | weight tensor manifest (mlc-llm name) |
License
Apache 2.0 (inherited from Qwen3-4B-Instruct-2507 base model).
