You can deploy this model and start earning money today!

Mahesh111000/
hanabi-qwen3-32b-lora-step70

Name: hanabi-qwen3-32b-lora-step70
Availability: InStock

Views

No views yet

Hanabi Qwen3-32B LoRA — step 70 (RL / GRPO via prime-rl)

RL training state for resuming on another machine. Base model: Qwen/Qwen3-32B (bf16).

Note: 97.5% of checkpoint_step_70/ is the frozen base model stored as fp32 master weights (32.76B x 4 bytes = 131 GB). Only ~3.2 GB is unique. If you don't need the exact optimizer state and step counter, just use adapter/ (1.07 GB) with a fresh optimizer — it holds all the learning, and it lets you change the learning rate (a resume does not).

path	size	purpose
`adapter/`	1.1 GB	LoRA adapter (r=32, alpha=64). The entire trained delta. PEFT/vLLM loadable.
`checkpoint_step_70/`	126 GB	prime-rl DCP checkpoint — model + optimizer + scheduler, exact resume.
`code/`	small	the custom reward env, config, and launcher (not on PyPI — required).

Training state at step 70

Checkpoint saved after step 69 completed; resuming runs step 70 next.
LoRA r=32, alpha=64, lr 1e-5, all 7 linear projections.
batch 256 = 16 problems x 16 rollouts, max_tokens 26624, seq_len 32768.
GPUs: 4 inference (TP=4) + 4 trainer (cp=4), 8x A100-SXM4-80GB.
Reward = deduction_score + move_reward (range [0,2]).

Observed at steps 65–69

step	reward	grad norm	entropy	mismatch KL
65	1.4244	0.0008	0.5994	0.0029
66	1.3072	0.0007	0.5936	0.0029
67	1.4862	0.0007	0.6049	0.0030
68	1.4189	0.0009	0.6000	0.0030
69	1.3700	—	—	—

Reward is flat within noise over this window — no trend either way. is_truncated = 0.0 throughout, so nothing is hitting the 26,624-token cap. copy_rate ~0.55 and flat, i.e. the degenerate "copy the previous belief state" policy is not being learned.

The small grad norm is expected, not a fault: the loss is token-mean-reduced over ~3.9M tokens per step and advantages are raw within-group deviations (no std normalisation), while Adam is scale-invariant so updates remain ~lr-sized.

Resume on a new server

bash

git clone https://github.com/PrimeIntellect-ai/prime-rl.git
cd prime-rl && git checkout 16e747c25e81c5c90bd860c68a588f7b151353a6
uv sync

huggingface-cli download Mahesh111000/hanabi-qwen3-32b-lora-step70 --local-dir ./hanabi-state

# custom reward env (required — not on PyPI)
uv pip install --python .venv/bin/python --no-deps -e ./hanabi-state/code/hanabi-deduction

cp ./hanabi-state/code/rl_32b.toml configs/hanabi_deduction/
mkdir -p outputs/checkpoints
cp -r ./hanabi-state/checkpoint_step_70 outputs/checkpoints/step_70

VLLM_ATTENTION_BACKEND=FLASHINFER \
  uv run rl @ configs/hanabi_deduction/rl_32b.toml --ckpt.resume-step 70

Dataset: Mahesh111000/Hanabi-init-30turns (or point dataset_path at a local copy).

Expect: the first step after any (re)start is a cold start while the async pipeline refills (Async Level climbs from -2 to 0) — measured 104 min, then 88, then settling to ~74–88 min per step. The trainer needs ~40 min and hides entirely inside the rollout phase, so step time is inference-bound. Needs 8x80 GB GPUs.

Full code, configs and profiling tooling: https://github.com/kaousheik-26/hanabi_latest_train_32b

Gotchas baked into the config (learned the hard way)

dist_timeout_seconds = 18000 — the default 600s kills the trainer on the cold-start step.
[orchestrator.client] timeout = 7200 — the default 1200s silently drops long rollouts.
ckpt.keep_last = 2 — without it, checkpoints fill the disk (~126 GB per save).
[trainer.tokenizer] name pinned — prime-rl otherwise resolves it from a stale default.
Resume always inherits the checkpoint's LR; the config's lr is ignored on resume. To change the learning rate you must start fresh from merged weights, not resume.
prime-rl's outputs/weights/ are NOT merged (bitwise identical to base) — don't rely on them. clean_lora_state_dict() (trainer/lora.py:220) discards the adapter. Use merge_lora.py.
A checkpoint named step_N is the state to resume at N, i.e. saved after step N-1 finished.
vLLM prefix caching is inert here: verifiers sets prompt_logprobs=True, which makes vLLM set skip_reading_prefix_cache and bypass the cache entirely (0 queries, not 0 hits).

Mahesh111000/hanabi-qwen3-32b-lora-step70