Vanilla GRPO on top of the agentic SFT. HONEST RESULT: the SFT was already saturated on the reward tasks — the RL run early-stopped at step 110 and the model is metrically indistinguishable from its starting point.

Overview

Stage: GRPO (TRL, LoRA, merged) — vanilla baseline
Lineage: Base → CPT → ParamΔ → SFT v2 → GRPO (this model)
Method: TRL GRPO [grpo], G=8 rollouts, LR 1e-6, KL β=0.02, graded task-routed rewards (recall-weighted triage + verbatim extraction) [toolrl], reward-plateau early stop. Prompt set: 8,000 verifier-generated prompts (4k triage / 4k extraction, real Mars baselines).
Domain: fictional — Asterion Space Operations (24-satellite constellation: alert triage, anomaly investigation, subsystem ops) + reused Mars Express telemetry.

Training


Prompts	asterion-grpo-prompts-lfm2 — 8,000 (4k/family)
Rollouts / LR / KL	G=8, 1e-6 cosine, β=0.02
Early stop	reward plateau at step 110 (~epoch 0.05): mean reward ~1.25, best 1.387

Evaluation

Metric	Value	Note
alert pass@1 (v3)	0.304	SFT parent: 0.304 — identical
per-difficulty (v3)	1.00 / 0.154 / 0.00 / 0.00	identical to parent
extraction pass@1 (v3)	0.83	parent: 0.84

Eval is a held-out, deterministic verifiable harness (synthetic tasks); baseline = the pre-SFT ParamΔ model. See noval-corp/scripts/eval_agentic.py.

Intended use & limitations

Same as the SFT parent — this checkpoint adds nothing measurable and exists as the vanilla-GRPO baseline of the round. Prefer the SFT parent.

Limitations:

NO measurable gain over the SFT parent: the v2-style reward tasks were already saturated by the SFT (pass@1 0.986 on the v2 verifier) — RLVR needs headroom to buy anything.
The v3 capability gap (multi-alert L3/L4 = 0.00) is out-of-distribution for BOTH the SFT data and the GRPO prompts — reward cannot elicit what the policy cannot express; the fix is dataset extension, not more RL.

Innovations tested

Negative result, kept on purpose — documents that vanilla RLVR from a task-saturated SFT is a no-op — the round's RL budget moved to the 12B decision instead.

Usage

Use inside a tool-execution loop (the model emits calls; you run the tools and feed results back):

python

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("atenareply/lfm2.5-1.2b-asterion-agentic-grpo")
model = AutoModelForCausalLM.from_pretrained("atenareply/lfm2.5-1.2b-asterion-agentic-grpo")
TOOLS = [...]  # the 8 OpenAI-style tool schemas (see noval-corp/config/curation.py: TOOL_SCHEMA)
messages = [{"role": "system", "content": "You are an OMC operations assistant."},
            {"role": "user", "content": "ALERT on NPWD2531: 1.12A. Routine or genuine?"}]
inputs = tok.apply_chat_template(messages, tools=TOOLS, add_generation_prompt=True, return_tensors="pt", return_dict=True)
# parse <|tool_call_start|>[...]<|tool_call_end|>, execute, append a {"role":"tool",...} message, repeat.

Reference loop: noval-corp/scripts/eval_agentic.py::run_episode.

Citations

GRPO — Shao et al., DeepSeekMath (Group Relative Policy Optimization), 2024. arXiv:2402.03300
ToolRL — Qian et al., ToolRL: Reward Design for Tool-Use RL, 2025. arXiv:2504.13958

Card generated by noval-corp/scripts/gen_model_cards.py (standardized across the noval-corp model family).