You can deploy this model and start earning money today!

knightnemo/
nanowm-b2-rt1-abl-pred-epsilon-50k

Name: nanowm-b2-rt1-abl-pred-epsilon-50k
Availability: InStock

lerobot/fractal20220817_data

Views

No views yet

NanoWM-B/2 · RT-1 · Ablation: pred_name = epsilon

One of three checkpoints from the pred_target ablation on RT-1 fractal (epsilon-prediction arm). Each arm runs in its native schedule environment — cosine + ZTSNR for v and x, linear + no-ZTSNR for epsilon — so the comparison isolates the prediction target rather than handicapping any one of them.

Run identity

wandb: https://wandb.ai/better_guidance/nano-world-model-ablation/runs/8feu8roy
launcher: src/scripts/ablation/pred_epsilon.sh
collection: https://huggingface.co/collections/knightnemo/nano-world-model

Training setup

Key	Value
Architecture	NanoWM-B/2 (12 layers, d=768, patch=2, 158.6M params)
Dataset	RT-1 fractal (`lerobot/fractal20220817_data`)
Frames × resolution	4 × 256² → 4 × 32² latents (SD-VAE)
Context frames	1 (sequential / self-forcing scheduling)
Action injection	additive (7-dim continuous)
Steps	50,000
Batch	8/GPU × 8 × H20 = 64 effective
Optimizer	AdamW, lr 1e-4, wd 0.01, warmup 1000, grad clip 0.1 after 20k
Precision	bf16-mixed (params fp32), VAE fp32, `torch.compile` on
Seed	3407

Diffusion setup

Key	Value
pred_name	epsilon
noise_schedule	`linear` (DDPM-native, to avoid cosine+eps degeneracy)
zero_terminal_snr	false
timestep_sampling	logit_normal (SD3-style, μ=0, σ=1)
snr_gamma	5.0 (Min-SNR loss weighting)
diffusion_steps	1000 train · 250 DDIM sample
history_stabilization_level (inference)	0.02

Loading

bash

git clone git@github.com:knightnemo/nano-world-model.git
cd nano-world-model
huggingface-cli download knightnemo/nanowm-b2-rt1-abl-pred-epsilon-50k --local-dir ./ckpt

python

import sys
from omegaconf import OmegaConf
from safetensors.torch import load_file
sys.path.insert(0, "src")
from models import get_models

cfg = OmegaConf.load("ckpt/config.yaml")
cfg.experiment.infra.compile = False
model = get_models(cfg).eval()

state_dict = load_file("ckpt/model.safetensors")
model.load_state_dict(state_dict, strict=True)  # 0 missing / 0 unexpected