betaFeedback

WorksComputer Science

When Agents Play Differently: Localizing Human–Agent Behavioral Divergence with Conditional Distributional Testing

Abstract

Evaluating whether artificial agents behave like humans requires comparing not just average outcomes but entire behavioral distributions across diverse contexts. We propose a stratified framework that computes nonparametric discrepancy statistics within contextual cells, and discovers subgroup rules identifying where agent behavior departs from human play. We show that session-level metrics dominated by win/loss outcomes offer little signal beyond mean comparisons, whereas process-level metrics such as move efficiency reveal genuine behavioral differences. Applied to a commercial match-3 game, it uncovers patterns invisible to aggregate metrics: a playtesting bot is diverging substantially from humans in certain game modes, while showing no detectable divergence in others. We release our implementation as open source to support reproducibility and adoption.

Paper details

vlnc:W_38qv7fue
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC-BY-NC-SA-4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…