An RL agent that doesn't just detect contamination — it infers the hidden causal intervention behind it. Trained via adversarial self-play, where an adversary learns to hide better as the investigator learns to reason better.

🔗 Quick Links

Resource	Link
🚀 Live Demo	HF Space
🤖 Trained Model	ms-shamanth/recalltrace-investigator
📓 Training Script	train_trl.py (Unsloth + TRL)
📊 Self-Play Training	run_selfplay.py

🎯 Problem: Why This Matters

Real-world supply-chain recalls (FDA food safety, automotive parts, pharmaceuticals) involve tracing contamination through complex multi-hop logistics networks — where evidence is partial, labels are unreliable, and bad actors actively conceal the source.

Current LLMs and RL agents struggle with:

Causal inference under partial observability — 30-50% of graph edges are hidden
Adversarial robustness — the contamination strategy adapts to the investigator
Belief calibration — knowing when you have enough evidence to quarantine

RecallTrace is the first OpenEnv environment that trains an agent to perform abductive causal reasoning against an adaptive adversary.

🌐 The Environment

What the Agent Sees

A supply-chain graph with nodes (warehouses, crossdocks, retailers) holding inventory lots. A recall notice alerts the agent to contamination — but the source, spread pattern, and intervention type are hidden.

What the Agent Does

Action	Purpose	Reward
`inspect_node`	Examine a node's inventory and evidence	+0.08 to +0.20
`trace_lot`	Follow a lot through the shipment graph	+0.12 to +0.25
`quarantine`	Isolate contaminated stock at a node	+0.28 (correct) / -0.35 (false positive)
`notify`	Alert downstream stakeholders	+0.04 per affected node
`finalize`	Submit final containment decision	Composite score (0-1)

What Makes It Hard

Hidden interventions: The adversary picks one of 3 strategies (lot relabeling, mixing events, record deletion) and places it in the graph
Decoys: False positives are planted to mislead the investigator
Partial observability: The agent must reason about hidden edges and infer causality
Adversarial curriculum: The adversary adapts its strategy based on agent weaknesses

🚀 Training

Self-Play Training (Heuristic Agents)

python run_selfplay.py

Runs 200 episodes in <2 seconds on CPU. The investigator and adversary co-evolve:

Figure 1: Four-panel training curves showing F1 improvement from 0.58 → 1.0, adversary reward declining, quarantine precision increasing (8.3 → 3.1 nodes), and investigation efficiency improving (25 → 11 steps).

Figure 2: Side-by-side comparison of untrained (spray-and-pray) vs trained (precision targeting) agent behavior on the same supply-chain graph.

LLM Training (Unsloth + TRL)

bash

pip install unsloth "trl>=0.12" datasets accelerate
python train_trl.py --push-model

Fine-tunes Qwen2.5-0.5B-Instruct (4-bit via Unsloth) on expert demonstrations using TRL SFTTrainer:

Data Generation: Runs heuristic expert on 300 episodes → collects high-reward (observation, action) pairs
SFT Training: Fine-tunes with LoRA (r=16) for 3 epochs
Evaluation: Compares random baseline vs heuristic vs trained LLM
Push: Uploads trained model to HF Hub

Re-run in Colab:

bash

!pip install unsloth "trl>=0.12" datasets
!git clone https://huggingface.co/spaces/ms-shamanth/recalltrace-openenv
%cd recalltrace-openenv
!python train_trl.py

📊 Results

Self-Play Performance

Metric	Early (ep 1-20)	Late (ep 181-200)	Improvement
F1 Score	0.576	1.000	+73.6%
Nodes Quarantined	8.3/episode	3.1/episode	-62.7%
Steps to Finalize	25.4	10.8	-57.5%
Quarantine Threshold	0.000	0.550	Learned selectivity
Exploration Rate	0.950	0.050	Learned focus

Key Insights

Spray-and-pray → Precision: Early agent quarantines everything; trained agent targets only confirmed contamination
Adversary co-evolution: Adversary shifts from lot relabeling (35%) to record deletion (35%) as investigator learns to handle relabeling
Belief calibration: Agent learns to only quarantine when P(contaminated) > 0.55, avoiding false positives

🧠 Why This Is Unique

Theme 3.1 — World Modeling

The agent maintains a probabilistic belief state (P(contaminated) per node) and only quarantines when confidence exceeds a learned threshold. This is world modeling — the agent builds an internal representation of hidden graph structure.

Theme 4 — Recursive Skill Amplification

Adversarial self-play creates an automatic difficulty curriculum. Both agents improve simultaneously: the adversary finds harder hiding spots, forcing the investigator to develop more sophisticated causal reasoning. This is recursive amplification — each improvement in one agent drives improvement in the other.

⚙️ How It Works

┌──────────────────────────────────────────────────────────┐
│                    Self-Play Loop                        │
│                                                          │
│  Adversary ──→ picks intervention type + placement       │
│      │                                                   │
│      ▼                                                   │
│  Environment ──→ generates contaminated supply chain     │
│      │                                                   │
│      ▼                                                   │
│  Investigator ──→ inspect, trace, quarantine, finalize   │
│      │                                                   │
│      ▼                                                   │
│  F1 Score ──→ updates both agents                        │
│      │                                                   │
│      └──→ repeat for N episodes                          │
└──────────────────────────────────────────────────────────┘

ms-shamanth/recalltrace-investigator

Uploaded model

RecallTrace: Causal Inference via Adversarial Self-Play