End-to-end recipe for training a character (or style) LoRA on Krea-2-Raw using musubi-tuner (kohya-ss), verified on a single-session full run on 96GB VRAM (RTX PRO 6000, Blackwell sm_120).

Showcase: QJ character LoRA on Krea-2 Turbo, throne scene with nebula skies

Above: one preview generated with this recipe's LoRA on Krea-2-Turbo. The character is "Queen Jedi (QJ)" — a private VRChat avatar of mine. See the Showcase gallery below for more.

What this is — and what it isn't

This is a training recipe, not a model release. What you get:

The full training pipeline, parameters, and scripts used in a real session.
Configuration templates (configs/) you adapt to your own data.
Lessons learned from the run, with measured numbers.

What is NOT here:

❌ The trained LoRA file (.safetensors) — kept private.
❌ The training dataset — kept private (photos of my own VRChat character "Queen Jedi").
❌ A way to download anything you can plug into ComfyUI directly.

The preview images in samples/ (if present) are a showcase of what this exact recipe produces with proper data. They aren't reproducible without your own dataset, but they tell you whether the recipe is worth your time.

Who this is for

You have ~96GB VRAM available: RTX PRO 6000 / H100 80GB+swap / H200 / A100 80GB (with caveats) / DGX Spark.
You have a coherent dataset (200-500+ images, 1:1 .txt captions).
You want a character LoRA that holds identity through scenes and poses on Krea-2-Raw.

For lower VRAM (24-48GB), see the Lower VRAM section — you trade speed for memory via --gradient_checkpointing and --fp8_base --fp8_scaled.

TL;DR

bash

# 1) Drop your dataset (images + 1:1 .txt captions) into a folder, e.g.:
#    ~/musubi-tuner/datasets/MY_CHAR/

# 2) Adapt configs (paths, batch_size, trigger token):
nano configs/dataset_example.toml
nano configs/sample_prompts_example.txt

# 3) Pre-cache latents + text encoder outputs (once):
bash scripts/precache_krea2.sh

# 4) Full run (16 epochs):
bash scripts/run_krea2_train.sh
tail -f ~/musubi-tuner/output/MY_RUN/run_*.log

# 5) Plot loss (any time, raw + 100-step moving average):
python scripts/plot_loss.py

# 6) Resume from last state if interrupted:
bash scripts/run_krea2_train.sh resume

Hardware requirements

Component	Tested with	Notes
GPU	RTX PRO 6000 (96GB VRAM, sm_120 Blackwell)	80GB cards also work; <80GB needs adjustments
RAM	128GB	`--turbo_dit_cache` parks ~25GB Turbo weights in CPU RAM
Disk	~80GB free	Models (~58GB) + dataset cache + checkpoints
OS	Linux	Ubuntu 24.04 OEM kernel 6.14 tested
Python	3.10+	Use a clean venv, isolate from any ComfyUI env

Stop any GPU process before launching — the trainer wants the entire VRAM budget.

Models you need

You need four files. The trainer is musubi-tuner (clone separately).

File	Source	Size	Notes
Krea-2-Raw weights	gated `krea/Krea-2-Raw` → `raw.safetensors`	25 GB	Requires HF token (gated repo)
Krea-2-Turbo weights	`krea/Krea-2-Turbo` → `turbo.safetensors`	25 GB	Optional but recommended for fast training-time previews
Qwen3-VL-4B text encoder	`Comfy-Org/Qwen3-VL` → `text_encoders/qwen3vl_4b_bf16.safetensors`	8.3 GB	Pre-packaged Comfy single-file build
Qwen-Image VAE	`Comfy-Org/Qwen-Image_ComfyUI` → `split_files/vae/qwen_image_vae.safetensors`	243 MB	Note: the repo is `Qwen-Image_ComfyUI`, not `Qwen-Image-Edit_ComfyUI` (the latter 404s)

Use hf download (the new HuggingFace CLI), not curl — multipart-parallel gets ~35 MB/s, plain curl often throttles below 5 MB/s on large HF files:

bash

HF_HUB_ENABLE_HF_TRANSFER=1 \
  hf download krea/Krea-2-Turbo turbo.safetensors \
  --local-dir ~/musubi-tuner/models/krea2/

For gated krea/Krea-2-Raw, set HF_TOKEN env first.

Dataset and captions

Drop your images + matching .txt caption files (1:1, same basename) into your dataset folder. Example: cat.png + cat.txt.

Caption rule that matters: describe only what is visible in the frame.

Close-up portrait → don't mention the full outfit, shoes, tail.
Full-body shot → describe the full outfit including trigger tokens.
Same logic for environment, props, accessories.

This trains the model to bind trigger tokens to visible attributes, not just to the character's full description. It's the difference between a LoRA that holds together across compositions and one that smears the trigger across the canvas.

Regularization images (optional but recommended for character LoRAs): a second [[datasets]] block in the TOML, mixing in 1:1 with the main set, helps prevent overfitting on subject features.

See configs/dataset_example.toml for the full template.

Sample prompts (training-time previews)

configs/sample_prompts_example.txt — one prompt per line, with CLI overrides at the end. The launcher uses --sample_every_n_epochs 1 --sample_at_first, so you get a 5-image preview grid every epoch.

Turbo-style preview overrides (matches --turbo_dit flag in the launcher):

--w 1024 --h 1024 --s 8 --l 1 --d 42

--s 8 — 8 sampling steps (Turbo)
--l 1 — CFG off (Turbo requires this)
--d N — fixed seed per prompt (stable identity across epochs)
--mu 1.15 — implicit from --turbo_dit, no need to pass

If you run previews through Raw (without --turbo_dit): use --s 28 --l 5.5 --d N. Raw at CFG=1 produces intentionally blurry images by design — this is not a bug.

Pre-cache (run once)

Two stages baked into scripts/precache_krea2.sh:

Latents — VAE encodes every image into cache/<name>_<res>_kr2.safetensors.
TE outputs — Qwen3-VL encodes every caption into <name>_kr2_te.safetensors.

After this, the trainer doesn't load VAE/TE at each step — the cache is on disk, training is pure DiT forward/backward. Typical throughput: ~15 it/s for the precache pass, ~1 minute for 800 image-caption pairs.

Re-run only if the dataset or captions changed.

Training launch

scripts/run_krea2_train.sh [mode]. Modes:

Mode	What it does
(no arg) or `run`	Start in `tmux` session `krea2` (falls back to foreground if no tmux)
`resume`	Pick up the last `*-state/` checkpoint from output
`dryrun`	50-step sanity run plus `--sample_at_first`, session `krea2_dry`
`fg`	Foreground, no tmux (for debug or invoking from your own background shell)

The launcher embeds these tuned parameters (measured on real runs):

Group	Flag	Why
Speed	`batch_size=1` (in TOML), no `--gradient_checkpointing`	Measured 1.22 s/it — 3.8× faster than bs=2+gc
Memory	`PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True`	Anti-fragmentation, headroom for sample-cycle spikes
Optimizer	`--optimizer_type adamw --learning_rate 1e-4`	Full fp32 — at 96GB VRAM, 8-bit optimizer isn't worth the complexity
Flow-matching	`--timestep_sampling shift --discrete_flow_shift 2.5`	Krea-2 recommended for 1024×1024
LoRA	`--network_module networks.lora_krea2 --network_dim 32 --network_alpha 32`	Krea-recommended default; covers all 264 Linear layers
Attention	`--sdpa`	Default, no extra deps. `--flash_attn` or `--sage_attn` work too if you've built them for your arch
Save + sample	`--save_every_n_epochs 1 --sample_every_n_epochs 1 --save_state`	Synchronized: checkpoint + preview at the same point. Can switch to `--save_every_n_steps N`
Storage	`--save_last_n_epochs 10 --save_last_n_epochs_state 10`	Keep the last 10 LoRA `.safetensors` + 10 state dirs; older are auto-purged
Previews	`--turbo_dit + --turbo_dit_cache`	Turbo parked in CPU RAM (~25GB), swapped into VRAM only during sample cycles
Logs	`--logging_dir ~/musubi-tuner/logs --log_with tensorboard`	Use `scripts/plot_loss.py` or `tensorboard --logdir ~/musubi-tuner/logs`

Monitoring

bash

# Live log
tail -f ~/musubi-tuner/output/MY_RUN/run_*.log

# GPU
watch -n 1 nvidia-smi

# Loss plot (raw + 100-step moving average → PNG):
python scripts/plot_loss.py

# Or interactive TensorBoard:
tensorboard --logdir ~/musubi-tuner/logs --port 6006
# open http://localhost:6006

Preview filenames: <name>_e<epoch>_<promptIdx>_<timestamp>_<seed>_<batchIdx>.png. Epoch tells you when; seed tells you which prompt (42-46 for the default 5).

Resume

State is written to output/MY_RUN/MY_RUN-e<N>-state/ at every save checkpoint. The launcher's resume mode picks the latest state automatically:

bash scripts/run_krea2_train.sh resume

To resume from a specific state (not the latest):

bash

accelerate launch \
  <musubi-tuner>/src/musubi_tuner/krea2_train_network.py \
  <all the usual flags from the launcher> \
  --resume <musubi-tuner>/output/MY_RUN/MY_RUN-e000005-state

Optimizer, scheduler, and LoRA weights all restore exactly — you continue from the same step with the same internal state.

Memory — measured, not theoretical

On RTX PRO 6000 96GB VRAM, dry-run measurements:

Config	s/it	VRAM peak	Verdict
bs=2, gc=off	—	94 GB	❌ OOM (forward activations don't fit)
bs=2, gc=on	4.7	38 GB	✅ Stable, 58 GB headroom
bs=1, gc=off	1.22	~55-65 GB	✅ Speed winner
bs=1, gc=off + expandable_segments	1.22	~55-65 GB	✅ Adopted as default

Krea-2 is a big DiT (features=6144, 28 wide blocks, multiplier=4). At bs=2 without gradient checkpointing, forward+backward activations alone are ~60GB.

Choosing your config:

Speed > gradient smoothness → bs=1, gc=off (default; ~4.5h for 16 epochs)
Smoothest gradients → bs=4 + gc on (theoretically ~5.5h)
OOM anywhere → enable --gradient_checkpointing (−30% speed, −50GB activations)

Bucket aspect-ratio impact (practical, not in official docs)

A side observation from running this recipe on two different datasets:

Dataset shape	Buckets seen	s/it
Square-only (all 1024×1024 after bucketing)	1 bucket	1.20
Mixed aspects (1248×832 + 880×1184 + 1184×880)	3 buckets	1.40-1.46

A ~15-17% wall-time penalty comes from bucket re-allocation when training batches alternate between shapes. The activations get re-laid-out, the cuDNN kernels re-pick algos, and you pay for it on every transition.

Actionable: if you want maximum throughput, normalize your dataset to a single aspect ratio (crop / pad / square-up) before training. The trainer won't complain about mixed aspects, but you do pay for the variety.

Not a deal-breaker for character LoRAs — variety in poses and crops is more valuable than 17% off the clock. But worth knowing.

Real-world measurements (full 16-epoch run, 2026-06-26)

A real run (not dry-run), 474-image main set + 348 regularization, bs=1, gc=off:

Metric	Value
Speed	1.20 s/it steady-state
VRAM steady	72 GB / 97 GB (24 GB headroom)
GPU utilization	99%
Temperature	77 °C
Power	500 W
Total wall time	4h 25m training + 16 sample cycles ≈ ~4h 36m
Total steps	13,152 (822 steps/epoch × 16)
Final loss/epoch	0.0227 (started at 0.0312 on epoch 1, monotonic decrease)

Dry-run showed 55-65GB; the full run sat at 72GB. The difference is most likely --turbo_dit_cache keeping mappings/buffers around in VRAM even when the Turbo weights themselves are parked in CPU RAM. The 24GB headroom held through all 16 epochs and all sample cycles — no OOM.

Smaller-dataset datapoint

A second run on the same hardware with a ~6.5× smaller dataset, to give readers a rough scaling sense for their own work:

	Run A (large dataset)	Run B (small dataset)
Image-caption pairs	~800 (square 1024×1024)	127 (mixed aspects)
Steps per epoch	822	127
Epochs	16	16
Total steps	13,152	2,032
Speed	1.20 s/it	1.40–1.46 s/it
Wall time — training	~4h 25m	~50m
Wall time — total (incl. 16 sample cycles)	~4h 36m	~1h 13m
Final loss/epoch	0.0227	0.0522

Same config across both runs: bs=1, gradient_checkpointing off, AdamW 1e-4, sdpa, network_dim 32 / alpha 32, 16 epochs, save_every_n_epochs 1, sample_every_n_epochs 1.

The higher final loss on Run B is expected — fewer total steps means training stops sooner, before the loss curve has fully flattened. Quality is still solid in practice (identity holds, character stacks cleanly with style LoRAs) — final loss is a useful indicator but not the only one. If your dataset is in the 100-200 image range, plan for roughly 1 hour of wall time on 96GB and expect a slightly higher loss floor than the larger-dataset run; both can produce strong LoRAs.

Troubleshooting

Symptom	Cause	Fix
loss=0 at start, no decrease	Wrong `timestep_sampling` (linear instead of shift)	Pass `--timestep_sampling shift`
OOM on forward training step	`gradient_checkpointing` off + batch > 1	Enable `--gradient_checkpointing` or use batch=1
OOM when loading Turbo at `--sample_at_first`	Partial Turbo file (size < 20GB)	Re-download with `hf download`; verify size before launching
`sample_at_first` hangs for 30-60s	`--turbo_dit_cache` quantizes 25GB once	Expected, subsequent sample cycles are fast
Raw previews are blurry	CFG=1 on Raw is intentionally blurry (per docs)	Use `--turbo_dit` for previews, or pass `--l 5.5 --s 28` to Raw
Final `avr_loss` plateaus around epoch 14-15	Diminishing returns near the model's training-loss floor	Use the e14 or e15 LoRA checkpoint — the auto-purge keeps the last 10

Lower VRAM (24-48GB)

This recipe is tuned for 96GB. For smaller GPUs, the path is:

48GB (RTX 6000 Ada, A6000): --gradient_checkpointing + bs=1 fits with margin.
24GB (RTX 4090/5090): --gradient_checkpointing + bs=1 + --fp8_base --fp8_scaled (halves DiT to ~12.5GB). Expect quality variance vs. fp16/bf16 base.
<24GB: offload more (CPU offloading via Accelerate config) or accept unreasonable wall times. Krea-2 is just a big model.

I haven't measured these myself — if you run them, a follow-up issue / PR with your measurements is welcome.

Repository layout

.
├── README.md                            # this file
├── LICENSE                              # CC BY 4.0
├── .gitattributes                       # Git LFS rules (images / safetensors)
├── scripts/
│   ├── precache_krea2.sh                # one-time latent + TE cache builder
│   ├── run_krea2_train.sh               # main launcher (run / resume / dryrun / fg)
│   └── plot_loss.py                     # TB events → raw + MA-100 PNG
├── configs/
│   ├── dataset_example.toml             # neutral dataset config template
│   └── sample_prompts_example.txt       # neutral preview prompts template
└── samples/                             # showcase preview images (CC BY 4.0)

Scripts contain hardcoded paths from the original setup (/home/jahjedi/musubi-tuner/...). Adjust them to your environment before running.

About the showcase

The preview images in samples/ come from training this recipe on a private dataset of my own VRChat character "Queen Jedi (QJ)" — a demon-queen persona with purple skin, blonde hair, curved horns, a floating crown, and a tail. I own both the avatar and the photographs used as training data.

Neither the dataset nor the trained LoRA are part of this repository. The previews are here only to show what this recipe can produce; reproducing them requires your own data.

Each image was generated with the trained QJ LoRA stacked on top of Krea-2-Turbo, often combined with a community style LoRA at strength 0.6-0.8. The QJ LoRA held identity consistently at 0.8 weight even under heavy style LoRA mixing — a sanity check that the recipe is not over-fitting.