Fine-tuned for the Meta x Hugging Face OpenEnv Hackathon (April 2026).
This adapter is trained to reduce authority bias in incident triage: the agent should prioritize runbook evidence and causal logs over contradictory social pressure in Slack-style messages.

Model Details

Base Model: Qwen/Qwen2.5-0.5B-Instruct
Adapter: QLoRA / LoRA adapter (Shiggii/qwen-incident-response-grpo)
Training Framework: TRL 1.2.0, GRPO
Training Steps: 384 optimizer steps
Compute: Kaggle free tier (Tesla T4), <10 GPU-hours total
Hackathon Context: OpenEnv environment-focused benchmark development

Training Philosophy

We optimized for iteration speed and environment quality over brute-force model scale:

Fast prototyping loop for reward/environment design
Lightweight fine-tuning footprint suitable for commodity hardware
Reproducible setup with public environment + notebook + artifacts

This was the final training iteration after API-based prototyping with llama-3.1-8b-instant to validate task and reward design.

Environment and Task

Environment:
https://huggingface.co/spaces/Shiggii/incident-response-detective

The model acts in a synthetic but realistic incident-response environment where each step includes:

Logs (timestamped system events)
Chat history (social signal, can be adversarial)
Runbook (official procedure constraints)

Action Space

The model selects one remediation action from:

rollback_deployment
scale_infrastructure
flush_redis_cache
notify_cto
restart_api_gateway
rotate_db_credentials
enable_circuit_breaker
purge_cdn_cache

Reward Logic (Environment Ground Truth)

Reward is shaped around:

Safety (runbook compliance / avoiding dangerous actions)
Efficiency (how quickly the incident is resolved)

Dangerous actions are heavily penalized; correct first-step resolutions receive near-ceiling scores.

Performance

Evaluation during prototyping showed the key behavioral gain on authority-bias scenarios:

Difficulty	Untrained Baseline	After Training	Improvement
Easy	0.201	0.999	+397%
Medium	0.999	0.999	--
Hard	0.999	0.999	--

Key finding: untrained behavior on Easy adversarial tasks reflects authority bias (following confident chat over runbook/log evidence). After GRPO fine-tuning, the model consistently follows evidence and documented procedures.

Training Curves and Provenance

Training/eval metrics are tracked in trainer_state.json (log_history, 384 entries)
Aggregate progression (early vs late): 0.946 → 0.995 mean reward
Plots are regenerated from authoritative training artifacts via:
- python regenerate_plots.py

Related artifacts are documented in the environment repo.

Reproduction

Training Notebook (Kaggle):
https://www.kaggle.com/code/shikharkumarsanjay/notebookb5136cd284
Environment (Space):
https://huggingface.co/spaces/Shiggii/incident-response-detective
Training metrics artifact: trainer_state.json (model repo)

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_id = "Qwen/Qwen2.5-0.5B-Instruct"
adapter_id = "Shiggii/qwen-incident-response-grpo"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto" if torch.cuda.is_available() else None,
)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()