This is an experimental ad policy compliance agent fine-tuned using Group Relative Policy Optimization (GRPO) via Unsloth and Hugging Face's TRL library. It is specifically designed to navigate the MetaGuard Ad Policy Sandbox, a multi-agent reinforcement learning environment where the model must gather signals and make terminal enforcement decisions.

Required Parser Patch

To successfully step the environment without crashing the API, you must sanitize the model's output before passing it to the server. Use the following Python drop-in replacement for your parsing logic. This guarantees the required reasoning field is present, enforces strict enums, and strips out hallucinated keys to perfectly satisfy the environment schema.

python

import json

def parse_attempt(text):
    candidates = []
    if "```" in text:
        for p in text.split("```"):
            p = p.strip().lstrip("json").strip()
            candidates.append(p)
    candidates.append(text.strip())
    s, e = text.find("{"), text.rfind("}") + 1
    if s != -1 and e > s:
        candidates.append(text[s:e])
        
    for c in candidates:
        try:
            r = json.loads(c)
            if isinstance(r, dict) and "action_type" in r:
                # 1. Guarantee required reasoning
                r.setdefault("reasoning", "No reasoning provided.")
                
                # 2. Enforce strict enums for violation_category
                valid_cat = {"HEALTHCARE", "FINANCIAL", "NONE", None}
                if r.get("violation_category") not in valid_cat:
                    r.pop("violation_category", None)
                    
                # 3. Nuke hallucinated extra fields
                allowed = {"action_type", "reasoning", "violation_category", "metadata"}
                return {k: v for k, v in r.items() if k in allowed}
        except Exception:
            continue
    return None

Evaluation Results

The model was evaluated across a 4-task suite testing healthcare policies, financial policies, multimodal analysis, and targeting constraints (10 steps per episode).

Baseline (Pre-GRPO)

Success Rate: 0/4 (0%)
Mean Reward per Step: -0.185

Final (Post-GRPO)

Success Rate: 0/4 (0%)
Mean Reward per Step: -0.050

Note: The GRPO fine-tuning successfully improved the agent's intermediate signal gathering phase, significantly pulling the mean reward up from the baseline.

Limitations

While the model shows strong capability in the information-gathering phase (e.g., calling query_regulations and analyze_image), it currently struggles with strict terminal sequence chaining. Specifically, the agent frequently fails to explicitly call the submit_audit action before attempting to issue a final approve or reject decision, resulting in an incomplete task execution.

Developed by: parth-1
License: apache-2.0
Finetuned from model : parth-1/metaguard-policy-agent-v1

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.