The SFT examples were produced from AutoResearch-style task factories with locked evaluators and scalar scorers. Passing runs were converted into clean instruction/solution records.

Training Metrics

Root baseline adapter:

Metric	Value
Global step	23
Max steps	23
Epoch	0.983957
Last train loss	0.3516595
Final eval loss	0.3447445
Final eval perplexity	1.41163

Initial QLoRA test adapter:

Metric	Value
Global step	50
Max steps	50
Epoch	0.787402
Last train loss	0.3274834
Final eval loss	0.3425751
Final eval perplexity	1.40857

These are internal SFT validation metrics only. They are not held-out browser-task pass rates.

Held-Out Evaluation Status

The next required evaluation is task-level generation against the locked held-out eval banks:

Three/R3F/WebGPU eval tasks
HyperFrames/Canvas/WebGPU eval tasks

Those evals should run the model-generated editable file through the existing Python, TypeScript, Playwright, canvas, and HyperFrames scorers. Until that is complete, this adapter should be treated as a first SFT baseline rather than a proven production coding model.

Usage

Example PEFT loading pattern:

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = "Qwen/Qwen3.6-27B"
adapter = "drainer/qwen36-threehf-sft-adapter"

tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, adapter)
model.eval()

messages = [
    {
        "role": "user",
        "content": "Return only a complete src/solution.tsx React Three Fiber scene using Drei OrbitControls.",
    }
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=2048,
        temperature=0.2,
        top_p=0.9,
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))

Serving With vLLM LoRA

Example:

vllm serve Qwen/Qwen3.6-27B   --enable-lora   --lora-modules qwen36-threehf=drainer/qwen36-threehf-sft-adapter   --max-model-len 16384   --dtype bfloat16

Then call the served model name qwen36-threehf.

Intended Use

This adapter is intended for experimentation with specialized code generation for browser graphics and 3D app-building tasks, especially tasks that output complete editable files for locked harnesses.

Good target outputs include:

src/solution.tsx for React Three Fiber task harnesses
index.html for HyperFrames/Canvas task harnesses
Compact, self-contained scene implementations
Deterministic visual compositions suitable for automated scoring

Limitations

This is a small first-pass SFT run, not a fully validated specialist model.
The adapter has not yet been benchmarked against the held-out browser execution evals.
It may overfit to the local task harness output style.
It may still generate invalid imports, incomplete files, or visually incorrect scenes.
It is not a merged model; the base model is required.
Use of this adapter is subject to the base model's license and terms.

Provenance

Project repo: https://github.com/trevor-ofarrell/three-autoresearch
Adapter repo owner: Trevor O'Farrell
Final metrics record on training machine: /workspace/data/three-autoresearch-training/first_run_final_metrics.json
Baseline reload check: passed; generated a valid-looking React Three Fiber sample from the adapter.