You can deploy this model and start earning money today!

GoodStartLabs/
opus-4b-cube-py-step120-2026-04-30

Name: opus-4b-cube-py-step120-2026-04-30
Availability: InStock

peft

safetensors

lora

reinforcement-learning

Views

No views yet

opus-4b-cube-py-step120

LoRA adapter for Qwen/Qwen3.5-4B, RL-trained on Opus Magnum-style motion-planning puzzles. The agent writes Python that places parts (arm, track, bonder, calcification glyph) and programs the arm to deliver atoms; the verifier scores solved=True when the program produces the target output.

Snapshot from training step 120 of the 4B-cube run.

Load it

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3.5-4B"
adapter = "GoodStartLabs/opus-4b-cube-py-step120-2026-04-30"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, adapter)

Or with vLLM / SGLang via --lora-modules.

Inference settings the model was trained for

These match the rollouts the model saw during training; deviating will degrade performance.

setting	value
Renderer	`qwen3_5_disable_thinking` (no `<think>` blocks; cookbook variant)
Coord mode	`cube` — 3-tuple `(x, y, z)` coordinates with `x+y+z=0`
Representation	`python` (model emits a Python program calling `place_arm`, `place_track`, `place_calcification`, `place_bonder`, `arm.grab/drop/rotate_/track_`, then `run()`)
Sampling temperature	1.0
Max tokens / turn	8192
Max trajectory tokens	40000
Max turns / episode	6 (multi-turn with verifier feedback between turns)

Training recipe (RL)

param	value
Algorithm	GRPO (importance-sampling loss)
LoRA rank	32
Learning rate	1e-5
Optimizer	Adam (β₁=0.9, β₂=0.95)
Group size	8
Groups per batch	16 (→ 128 rollouts/batch)
`num_substeps`	1
`kl_penalty_coef`	0
`temperature`	1.0
`remove_constant_reward_groups`	False
Reward	`+1.0` solve, `+0.1` valid-but-wrong submission (stacks), `-0.1` overflow, `-0.05` no-code on last turn

Task / data

Distances trained on: 1, 2, 3, 4 (hex distance from input to output cell)
Task types: move, transmute (bond was excluded — see history below)
Each batch samples 16 distinct puzzles × 8 rollouts = 32 trajectories on each of 4 puzzles… (well, groups_per_batch=16 distinct puzzles × group_size=8 rollouts = 128 rollouts/batch)
Puzzles drawn deterministically by seed across the run

Training history

This adapter is the latest checkpoint of a multi-phase run (resumed across phases due to a tinker incident and a bond-task ablation). Effective progression:

phase	batches	task_types	notes
1	0–60	move, transmute, bond	initial training, full task set
2	60–100	move, transmute (no bond)	bond removed (was 0% solve rate, pure compute waste)
3	100–120	move, transmute (no bond)	extension to step 150 (in flight at upload time)

Behavioral notes

The model has converged on a stereotyped recipe: place_arm → grab → rotate_cw/ccw → drop → reset. About half of solved trajectories also use place_track(...); tracks are productive when used.
Transmute tasks add place_calcification(...) before grab, off the rotation path.
Move tasks rarely use tracks at d1/d2; tracks become valuable at d3/d4.
One-shot solves (verifier returns solved=True on turn 1) account for the majority of successful trajectories on easy puzzles. On d3/d4 the model often needs 2-3 turns of verifier feedback (mostly correcting overlap-with-input or arm-pivot errors).

Approximate solve rate at this checkpoint (4B-cube run, late iters)

Trajectory-level (fraction of rollouts where verifier returned solved=True):

distance	rate
d1	~94%
d2	~99%
d3	~71%
d4	~76%

Move tasks ~91%, transmute tasks ~72%. The d3/d4 deficit is mostly in transmute, where the model occasionally gets stuck in track-overlap-with-arm-pivot loops.

Provenance

Base: Qwen/Qwen3.5-4B
Trained on Tinker; LoRA exported via tinker_cookbook.weights.{download, build_lora_adapter} and pushed via publish_to_hf_hub.
Source repo (training harness, prompts, verifier wrapper): the opus-lite-rl codebase.
Snapshot date: 2026-04-30