Views
No views yet
{"values":[3,7,9]}. Solving a puzzle = repeatedly querying the
policy cell-by-cell.Avra98/sudoku-latent-backtracking).| Stage | Target = values consistent under … |
|---|---|
| 1 | direct row/column/box constraints (depth-1 legal values) |
| 2 | + 2-step lookahead consistency |
| 3 | + 3-step lookahead consistency |
--stage_i, so each stage trains on its
own (and only its own) consistency set; a cell legitimately has multiple valid values.--full_finetune): all backbone parameters are trained (small models
lack the capacity to learn this through low-rank adapters), and checkpoints store the full model.--local_constraint_prompt): the prompt appends the explicit
per-cell row/column/box constraint block. This is essential for GPT-2 — without it the
models collapse to constant predictions; with it, GPT-2-large reaches perfect Stage-1 SFT.--num_cot_tokens k, recurrent_hidden): an optional recurrent
latent "thinking" step, introduced after a warm-up.k=0): no latent token. SFT (lc_large) reaches perfect Stage-1 solve; GRPO
refines from there.k=1): recurrent latent token switched on after warm-up.runs/local_constraint/): plain_* = no local constraint (collapses),
lc_* = local constraint, lat_lc_* = local constraint + latent. Finding: local constraint is
the decisive ingredient; the latent token adds optimization difficulty for the smallest models.cc/ # training/eval code (this experiment's code copy)
latent_multi_output_cell_policy/ # SFT + GRPO trainers
multi_output_cell_policy/ # prompt builder, rewards, shared policy
runs/
local_constraint/<variant>/checkpoint-step-04000/ # final SFT models (full model)
stage1_large/<baseline|latent>/{sft,grpo}/ # current Stage-1 runs
logs/ # training logs
code/ # launch + Hugging Face push scriptsmodel.safetensors) — load directly with
AutoModelForCausalLM.from_pretrained(<checkpoint dir>).1# Stage-1 gpt2-large, baseline (k=0) or latent (k=1): SFT -> GRPO
2bash code/stage1_large.sh baseline 0
3bash code/stage1_large.sh latent 1