betaFeedback

WorksMathematics

Conformal Selection for Hierarchical Streaming Data

Abstract

Many real-world tasks involve making sequential decisions about whether to select candidates before their responses of interest are revealed. In this work, we address the online selection problem in the context of hierarchical data, where candidates arrive sequentially from different populations. This extends existing online conformal selection frameworks beyond the standard i.i.d./exchangeable data assumption. The primary challenge lies in constructing valid statistics for each candidate sequentially under the hierarchical data structure. To this end, we first develop Online Hierarchical Selection (OHS), a distribution-free method that enables principled candidate selection for hierarchical data streams. Specifically, we introduce methods to construct rank-based e-values for both between-group and within-group samples, and apply averaged e-values to the online e-BH procedure. We then develop OHS with feedback (OHSF), which boosts the power of OHS by leveraging potential response feedback. Theoretically, the proposed methods achieve false discovery rate control under mild hierarchical exchangeability and data arrival assumptions at any time. We also generalize methods to broader settings, such as data with covariate shift and selection under effect-related responses. Extensive experiments on both simulated and real-world datasets demonstrate the effectiveness of the proposed methods.

Paper details

vlnc:W_hywz5myv
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…