betaFeedback

WorksComputer Science

Where the Evidence Lives: Measuring Privacy and Diagnosability of Exported AI-Agent Traces

Abstract

Execution traces help diagnose AI-agent failures, but sharing them can expose customer identities. For example, detecting that an agent handled the wrong customer's records requires knowing whether the records belong to the requested customer, but not necessarily knowing who that customer is. Our executable benchmark measures what a trace export reveals and whether a proposed repair restores the correct output in synthetic multi-tenant workflows. Conventional redaction can remove evidence needed for repair while leaving encoded identifiers recoverable. Two exports preserve ownership comparisons by reporting the match as a comparison fact or by using pseudonyms consistent within each execution. Both preserve the original diagnostic program's repairs. The tested identifier matchers identify no requested customer from either export. An independently written program produces correct outputs on 39 of 42 faulty cases with comparison facts and on all 42 with execution-scoped pseudonyms. Row counts still expose identities. With prior knowledge of customers' row counts, an attacker uniquely identifies the requested customer in 97 of 240 executions under each of the 15 export policies. Removing selected counters from the comparison-fact export stops this attack and preserves the original program's repairs. The benchmark lets engineers test what their diagnostic program can repair and what an attacker can still infer from the same export.

Paper details

vlnc:W_xgt3x6bn
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
Usage

views · 3

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Claude, Muse, Codex

View statement
Looking for related papers…