公开(公告)号：US20220283932A1

公开(公告)日：2022-09-08

申请号：US17192320

申请日：2021-03-04

Applicant: Adobe Inc.

Inventor： David Arbour ,  Ritwik Sinha ,  Ian Waudby-Smith ,  Aaditya Ramdas

IPC: G06F11/36 ,  G06F16/958 ,  G06F8/77

Abstract: A computer-implemented method includes instantiating a framework configured to optimize a metric of interest for a website based on interactions by participants with instances of a website in a controlled experiment. The instances of the website include one of two variants of digital content. Test data including an estimate of an effect on the metric of interest is generated based on the interactions. A sequence of confidence intervals is dynamically generated while the controlled experiment is ongoing. The true effect and the estimate effect on the metric of interest are both bounded by the sequence of confidence intervals throughout the controlled experiment. As such, an anytime analysis with anytime-valid test data is enabled while the controlled experiment is ongoing.