Telco Troubleshooting Agentic Challenge — Track B Submission

What's inside code.zip

track_b_submission/
├── README.md
├── run.sh / run.py                # entry point
├── requirements.txt
├── .env.example                   # backend configuration template
├── src/
│   ├── agent.py                   # Qwen critique-and-refine wrapper
│   ├── llm_client.py              # vLLM + OpenRouter client
│   ├── harness.py                 # classify, test loader, evidence reader
│   ├── network_model.py           # per-question network state
│   ├── parsers.py                 # vendor CLI parsers
│   ├── solver_fault.py            # fault root-cause solver
│   ├── solver_path.py             # L3 path solver
│   ├── solver_topo.py             # L2 forwarding-path solver
│   ├── llm_fault.py               # fault prompt + validator + taxonomy
│   ├── runtime_logger.py          # per-problem runtime decorator (spec)
│   └── sandbox_client.py          # live /get_devices_list and /execute
├── data/
│   ├── test.json                  # Phase 2 questions (100)
│   └── vendors.json               # static device roster + vendor map
├── models/
│   └── deploy.sh                  # vLLM launch script (Qwen)
└── result/                        # produced by `bash run.sh`
    ├── README.md
    ├── results.csv
    ├── traces.json
    └── runtime.json

Approach (generic, topology-agnostic agent)

1. Device discovery (3-tier, fault-tolerant): POST /ip/api/agent/get_devices_list → if unavailable, crawl the topology from the devices named in the question (BFS via LLDP neighbours / interface descriptions, validating each candidate by probing it) → static known-roster only as a last resort. All calls are Bearer-authenticated and serialized (one in-flight request per token; ≤500 calls/question).
2. Understanding (LLM): a router extracts the task type + parameters as structured JSON, with a roster-aware deterministic backstop (framing-agnostic; no regex classifier).
3. Deterministic compute engines do the actual reasoning — grounded entirely in live CLI output, so the LLM cannot hallucinate the result:
   • fault — taxonomy detectors (shutdown / routing-walk / OSPF / BGP / VXLAN / L3VPN / MAC / loopback / host-info), no cross-question baseline;
   • path — hop-by-hop L3 trace over routing tables (recovers faulted/locked tables), with L2-access insertion;
   • link/topology — reciprocity-resolved adjacency with canonical port names (from display current-configuration, bandwidth suffixes stripped).
4. Format rendering (LLM): the structured result is rendered into exactly the format the question specifies (separators, interface-name style, reason spelling), using only the computed facts.
5. Grounding gate + never-blank fallback: any answer line referencing a non-roster device is dropped; a grounded ReAct agent is the last resort.

• results.csv — one final prediction per scenario
• traces.json — every completion produced
• runtime.json — per-problem wall time via the spec runtime_logger

How to run (summary)