ANYTIME VALID CONFIDENCE SEQUENCE FOR RELATIVE INCREMENT IN ONLINE TESTS

    公开(公告)号:US20250022006A1

    公开(公告)日:2025-01-16

    申请号:US18220496

    申请日:2023-07-11

    Applicant: Adobe Inc.

    Abstract: A method, a system, and a computer program product for analyzing data collected during a randomized controlled experiment to determine an effect of variations of digital content. Determination of the effect includes execution of first and second testing sequences that prompt responses to first and second digital contents, respectively, from users. The testing sequences execute during a predetermined duration of time. Responses to the first and second testing sequences generate first and second test data, respectively. One or more confidence intervals for each first and second test data are generated at a randomly selected time during the predetermined duration of time. A testing metric indicating the effect of the second digital content over the first digital content is determined at the randomly selected time. The testing metric is determined at any time before expiration of the predetermined duration of time.

Patent Agency Ranking