phynics/

agentic-publication-protocol-dataset

Paired reader conversations and blinded evaluations comparing an Agentic Publication Protocol (APP) paper agent against a general repository-aware agent, on 11 quantum-physics papers. For each paper, a neutral reader asks the same scripted questions to both agents; the two transcripts are anonymized and scored by a blinded evaluator on accuracy, informativeness, grounding, and honesty (1-10).

Evaluator: Codex CLI, gpt-5.5, reasoning effort xhigh… See the full description on the dataset page: https://huggingface.co/datasets/phynics/agentic-publication-protocol-dataset.