This repository contains a PEFT LoRA/QLoRA adapter trained for action selection in a deterministic Python Risk-compatible environment. The adapter is intended to emit one strict JSON object that selects from the prompt's candidate actions.

Output Contract

The model-facing contract is exactly:

{"action_index": 0}

action_index refers to the compact prompt-action list, not directly to a raw environment action. Runtime code maps that index back to the full legal action and the environment validates it. Invalid JSON, missing action_index, out-of-range indices, and illegal actions must be logged and replaced by FallbackSafeAgent.

Training Summary

Base model: Qwen/Qwen3-4B-Instruct-2507
Method: Unsloth 4-bit LoRA/QLoRA SFT, adapter-only save
Training rows: 49,000
Held-out validation rows: 1,000
Epochs: 1.0
Sequence length: 2048
LoRA rank/alpha: 16 / 16
Train loss: 0.1535
Held-out eval loss: 0.1226
Hardware: NVIDIA RTX A5000

Training command:

HF_HOME=/workspace/.hf_home python -m training.train_sft_unsloth --dataset data/sft/risk_sft_stratified_50k.jsonl --model Qwen/Qwen3-4B-Instruct-2507 --out models/adapters/qwen3_4b_risk_sft --max-steps -1 --num-train-epochs 1 --limit-rows 0 --validation-split 0.02 --split-seed 3407 --eval-steps 1000 --logging-steps 50

Benchmark Results

Evaluation	Rows	Strict JSON	Valid Index	Teacher Match	Invalid
Base model fixed prompt set	100	0.000	0.000	0.000	100
Adapter fixed prompt set	100	0.850	0.850	0.820	15
Adapter held-out validation set	1000	0.779	0.779	0.722	221

Evaluation command:

HF_HOME=/workspace/.hf_home python -m training.benchmark_policy --dataset data/sft/risk_sft_stratified_50k_val.jsonl --model models/adapters/qwen3_4b_risk_sft --out data/prefs/benchmark_heldout.json --limit 1000 --seed 3407

Linked Artifacts

Training dataset: https://huggingface.co/datasets/clarkkitchen22/risk-rl-lab-sft
Benchmark data and metrics: https://huggingface.co/datasets/clarkkitchen22/risk-rl-lab-benchmark

Limitations

This is a research adapter for a simplified Risk-compatible environment. It is not a standalone base model and it is not a guarantee of optimal play. The environment should remain authoritative and validate every selected action.

clarkkitchen22/qwen3-4b-risk-sft-lora

Risk RL Lab Qwen3 4B SFT LoRA Adapter

Output Contract

Training Summary

Benchmark Results

Linked Artifacts

Limitations

clarkkitchen22/
qwen3-4b-risk-sft-lora