Seminar 2023-15-09

Exact Conditional Independence Testing and Conformal Inference with Adaptively Collected Data

Lucas Janson

Associate Professor

Department of Statistics

Harvard University

Date: Friday, September 15, 2023

Time: 11:00 A.M. – 12:00 P.M. Eastern Time

Location: Johnson Center, Room 325 Meeting Room A

Randomization testing is a fundamental method in statistics, enabling inferential tasks such as testing for (conditional) independence of random variables, constructing confidence intervals in semiparametric location models, and constructing (by inverting a permutation test) model-free prediction intervals via conformal inference. Randomization tests are exactly valid for any sample size, but their use is generally confined to exchangeable data. Yet in many applications, data is routinely collected adaptively via, e.g., (contextual) bandit and reinforcement learning algorithms or adaptive experimental designs. In this paper we present a general framework for randomization testing on adaptively collected data (despite its non-exchangeability) that uses a weighted randomization test, for which we also present computationally tractable resampling algorithms for various popular adaptive assignment algorithms, da ta-generating environments, and types of inferential tasks. Finally, we demonstrate via a range of simulations the efficacy of our framework for both testing and confidence/prediction interval construction. The relevant paper is https://arxiv.org/abs/2301.05365.

Lucas Janson is an Associate Professor of Statistics and Affiliate in Computer Science at Harvard University, where he studies high-dimensional inference and statistical machine learning.

David Kepplinger

Nicholas Rios