Qwen3-0.6B Schema Gym LoRA
A small LoRA feasibility adapter trained to improve first-pass bare JSON output
for two application contracts:
- a seven-field creative brief used by Dream Machine;
- a four-field allowlisted operational plan used by Safety Planner.
This is a hackathon experiment, not a general JSON model or a production
safety system.
Results
The base model and tuned adapter were evaluated greedily on the same first 100
held-out prompts. Evaluation rejects Markdown fences, extra keys, missing
fields, empty values, and duplicate JSON keys.
| Metric | Base | Tuned |
|---|
| First-pass schema compliance | 0/100 | 100/100 |
| Semantic preservation | 0/100 | 98/100 |
| Safety semantic preservation | 0/50 | 50/50 |
| Dream semantic preservation | 0/50 | 48/50 |
The two remaining failures were difficult Dream examples where valid,
schema-compliant JSON copied the wrong subject. The failures are retained in
the published metrics rather than repaired or removed.
Training
- Base checkpoint:
Qwen/Qwen3-0.6B
- Training load path:
unsloth/qwen3-0.6b-unsloth-bnb-4bit
- Dataset: 500 deterministic synthetic training examples
- Validation pool: 200 disjoint examples
- Modal GPU: NVIDIA L4
- Steps: 126
- Effective batch size: 8
- LoRA rank and alpha: 16
- Learning rate:
2e-4
- Seed:
150626
- Training time: 146.742 seconds
- Train loss: 0.52823
The dataset contains equal Dream and Safety task coverage, with standard,
difficult, adversarial, and repair categories. Train and validation prompts
have zero overlap.
Loading
1from peft import PeftModel
2from transformers import AutoModelForCausalLM, AutoTokenizer
3
4base_id = "Qwen/Qwen3-0.6B"
5adapter_id = "KwabsHug/qwen3-0.6b-schema-gym-lora"
6
7tokenizer = AutoTokenizer.from_pretrained(adapter_id)
8model = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
9model = PeftModel.from_pretrained(model, adapter_id)
Use the exact system and user contracts represented in the dataset. The
adapter has not been evaluated as a drop-in structured-output solution for
unrelated schemas.
Limitations
- Training data is synthetic and template-controlled.
- Only 100 of the 200 held-out examples were used in the final Modal run.
- The reported result is one deterministic run, not a multi-seed estimate.
- Semantic checks cover controlled fields, safety wording, action allowlists,
and escalation triggers; they do not prove broad factual correctness.
- Safety Planner outputs are benchmark artifacts, not professional guidance.
- The adapter was trained through a 4-bit Unsloth load path and has not yet
been benchmarked after publication from a clean environment.
Reproducibility
Run ID: full-20260615-101457-d99920d9
The source repository includes the deterministic generator, validation tests,
Modal training program, raw per-example metrics, and retrospective analyzer.
