Least-Privilege RL adapters
LoRA adapters for the least-privilege terminal-agent experiments. The adapters
target
Qwen/Qwen3.5-4B and use PEFT
LoRA rank 32, alpha 64, and dropout 0.
Recommended adapters
main/seed-1-checkpoint-1500: primary policy selected by the frozen internal
protocol before external-benchmark analysis.continuation/selected-A0-seed1-checkpoint-400: selected post-hoc corrective
continuation from the primary policy, trained on 200 corrective tasks
interleaved with 200 source-exact replay tasks.
Repository contents
The release contains 32 physical final experiment endpoints:
- three primary checkpoint-1500 seeds;
- the selected continuation-400 endpoint;
- 12 base-start reward-ablation endpoints (
A0, A2, A4, and B1, three
seeds each);
- 16 production-policy continuation reward-arm endpoints (
A0 seeds 2-5,
A2 seeds 1-5, A4 seeds 1-2, and B1 seeds 1-5).
A0 continuation seed 1 is the selected continuation adapter and appears once
physically; adapter_manifest.json records its logical alias. Intermediate
autosaves and optimizer states are intentionally excluded: they are not reported
model endpoints and would add roughly 70 GiB of redundant training state.
Every weight file is accompanied by its PEFT adapter_config.json. The manifest
records source campaign, arm, seed, step, byte size, and SHA-256 hash, and
SHA256SUMS.txt permits direct integrity checking.
Loading
1from transformers import AutoModelForCausalLM, AutoTokenizer
2from peft import PeftModel
3
4base_id = "Qwen/Qwen3.5-4B"
5repo_id = "tu101/least-privilege-model"
6
7tokenizer = AutoTokenizer.from_pretrained(base_id)
8base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
9model = PeftModel.from_pretrained(
10 base,
11 repo_id,
12 subfolder="main/seed-1-checkpoint-1500",
13)
The evaluation environment supplies the tool-interface system prompt and
executes tool calls through the host-authoritative broker; loading an adapter
alone does not reproduce the sandbox. Code, first-party task catalogs,
reproduction instructions, compact result records, and the manuscript are at
Astrowebdeving/least_privilege_grpo.
Result summary and limitations
The primary checkpoint improves frozen internal safe success from 64.36% to
98.86% and lowers over-privileged success from 4.56% to 0.45%. External transfer
is heterogeneous: MetaTool and FORTIS direct skill selection improve modestly,
whereas the primary policy is approximately tied with base on aggregate
ToolPrivBench and has category-specific regressions. The corrective continuation
reduces ToolPrivBench over-privileged selection from 44.85% at its parent to
38.05% while retaining frozen internal performance; with ToolPrivBench's
privilege-aware prompt it reaches 26.65%.
These are research adapters, not a security boundary or a guarantee that tool
use is safe. Use the broker, sandbox, and policy controls from the code release.
