MemPABench Simulator Judge

The simulator judge is an independent post-hoc evaluator. It judges a recorded PA-user interaction against the accepted script, active preference setting, and rubric. It does not steer the simulator, choose beats, read simulator-only blocks, or reveal a verdict to the PA.

It receives the transcript, visible tool/state evidence, declared context-attribute target, and frozen rubric; it returns the schema-defined judgment and rationale. Ground truth comes from accepted persona material, the 2-context by 14-attribute matrix, promoted scripts, and event overlays. The judge complements, rather than replaces, the simulator’s subjective session-end self-report.

See MemPABench_Judge_Endpoint_Pool and MemPABench_Evaluation_Metrics.