You can deploy this model and start earning money today!

syvb/
nla-qwen2.5-7b-L20-ar-matryoshka-sonnet46-v3

Name: nla-qwen2.5-7b-L20-ar-matryoshka-sonnet46-v3
Availability: InStock

safetensors

qwen2

nla

natural-language-autoencoders

qwen2.5

kitft/nla-qwen2.5-7b-L20-ar

finetune

apache-2.0

Views

No views yet

nla-qwen2.5-7b-L20-ar-matryoshka-sonnet46-v3 — v3 warm-start (bullets prompt)

Critic (AR) of the NLA Qwen2.5-7B (layer-20) pair, v3 warm-start: SFT from the base kitft/nla-qwen2.5-7b-L20-ar on Claude Sonnet-4.6 explanations in the v3 format.

v3 changes vs v2 (see the repo's experiment README):

The prompt actually says bullet points ("a list of bullet points, one per line"); output text is plain one-item-per-line (no literal "- " markers). The nla_meta.yaml sidecar carries the real trained template (v1/v2 shipped a stale tagged one).
AR: critic inputs token-truncated to K ~ U[1,120] (2% left full) — pre-calibrated for the RL-time uniform token truncation.
Intended as the starting point for the v3 RL phase (KL 0.03, uniform token truncation ~U[1,120]).

Warm-start data: v3 splits (v3/ folder). Hyperparameters: global batch 256, lr 2e-5→2e-6 cosine, warmup 50, 1 epoch.

syvb/nla-qwen2.5-7b-L20-ar-matryoshka-sonnet46-v3

nla-qwen2.5-7b-L20-ar-matryoshka-sonnet46-v3 — v3 warm-start (bullets prompt)

syvb/
nla-qwen2.5-7b-L20-ar-matryoshka-sonnet46-v3