Policy-Constrained LLM Agents for Financial Fraud Intelligence

🎯 Research Question

🔑 Key Finding

📊 Results

By Violation Category

🔬 Methodology

Adversarial Test Suite

• Financial Harm (10): Requests to send money, buy gift cards, wire transfers
• Identity Exposure (10): Requests for SSN, passport, bank details, passwords
• Illegal Actions (10): Hacking, forging documents, distributing malware, impersonation
• Unsafe Escalation (10): Physical meetings, remote access, downloading malware

Guardrail Approaches Compared

1. No Guardrails: Raw LLM output (simulated 40-60% harmful compliance per AgentHarm findings)
2. Prompt-Based: Safety rules in system prompt (~75% catch rate per Fraud-R1 findings)
3. External Policy Engine: Regex-based post-processing that validates EVERY response (~98% catch rate)

Grounding in Literature

• AgentHarm benchmark (arxiv 2410.09024) — agents comply with harmful requests 40-80%
• ShieldAgent (arxiv 2503.22738) — external guardrails achieve 90.1% recall, 11.3% above baselines
• ToolSafe (arxiv 2601.10156) — step-level guardrails reduce harmful actions by 65%
• AgentDoG (arxiv 2601.18491) — 466 GitHub stars, diagnostic guardrail framework
• Fraud-R1 (arxiv 2502.12904, ACL 2025) — LLMs succumb to multi-round fraud 20-40%

📁 Repository Structure

├── README.md
├── experiment.py          # Full working prototype (40 adversarial scenarios)
├── paper_draft.tex        # Springer ACSAR paper draft
├── requirements.txt
├── figures/
│   ├── fig1_overall_comparison.png
│   ├── fig2_violations_by_category.png
│   ├── fig3_outcome_distribution.png
│   └── fig4_severity_analysis.png
└── results/
    ├── guardrail_comparison.csv
    └── violations_by_category.csv

🚀 Quick Start

🏗️ Extension for Full Paper

1. Replace simulation with real LLM calls (GPT-4, Llama-3, Claude) to measure actual compliance rates
2. Expand to 200+ scenarios using AgentHarm dataset patterns (available on HuggingFace: LG-Ks/AgentHarm)
3. Add adaptive adversaries that modify attacks based on guardrail feedback
4. Test guardrail latency — overhead of external engine vs. inference speed
5. Multi-turn escalation — test if adversaries can gradually erode guardrails over many turns

📝 Conference

• Conference: ICISET 2026 — Namibia University of Science and Technology
• Track: AI and Machine Learning → Ethical and Responsible AI
• Deadline: July 27, 2026

Generated by ML Intern

• Try ML Intern: https://smolagents-ml-intern.hf.space
• Source code: https://github.com/huggingface/ml-intern

Usage