SFT of hamishivi/Qwen3.5-9B for self / dynamic context management in long-horizon agents. The model is trained to decide when and how to compact its own context (via an edit_context tool) while solving long-context agentic tasks.

Training data

Segmented self-compaction trajectories generated with a Codex/Claude-Code-style harness and split at compaction boundaries into independent SFT sequences:

Each dataset ships normal / medium / high compaction-effort splits; all six are mixed.

Recipe

Base model: hamishivi/Qwen3.5-9B (gated-delta hybrid)
Objective: assistant-only loss with prompt-length-gated masking (carried-over context is masked; only newly generated tokens after each compaction boundary are trained)
Max sequence length: 32,768
Epochs: 2 · global batch: 128 · LR: 2e-5 (linear, warmup 0.03) · bf16
Parallelism: DeepSpeed ZeRO-3 (no offload) + Ulysses sequence parallelism
Trainer: vendored open-instruct fork

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("osieosie/qwen3.5-9b-cligym-tmax-compaction-nc0.3-sft-e2")
model = AutoModelForCausalLM.from_pretrained("osieosie/qwen3.5-9b-cligym-tmax-compaction-nc0.3-sft-e2", torch_dtype="bfloat16", device_map="auto")