公开(公告)号：US20220066913A1

公开(公告)日：2022-03-03

申请号：US17380152

申请日：2021-07-20

Applicant: Coupang Corp.

Inventor： Ngoc-Lan Isabelle Phan ,  Jun Ye ,  Beibel Ye ,  Chul Seo

IPC: G06F11/36 ,  G06Q30/02 ,  G06F16/248 ,  G06F16/245

Abstract: A computerized method for controlled testing, comprising: providing a design specification for event data, the design specification including at least one of an event data source, an event data filter, and an event format; retrieving testing metrics; configuring testing events based on the design specification, by: retrieving testing events from the event data source specified in the design specification; filtering the retrieved testing events based in the design specification; and formatting the filtered testing events based on the design specification; generating analysis data by applying testing metrics to the configured event data; generating output data based on testing rules; and generating a user interface to display the output data.

公开(公告)号：US11621894B2

公开(公告)日：2023-04-04

申请号：US17307187

申请日：2021-05-04

Applicant: Coupang Corp.

Inventor： Jun Ye ,  Chui Seo

IPC: H04L41/22 ,  H04L41/14 ,  G06F40/205

Abstract: A system for live analysis of testing logs. The system including a memory and a processor configured to execute the instructions to perform operations including receiving a plurality of log entries; processing the plurality of log entries; storing the processed plurality of log entries in a database having an inverted index; receiving a query from a user device; returning test data, metadata, and statistics related to the one or more log entries; displaying the test data, the metadata, and the statistics on a GUI; comparing an exposure ratio to a range, the exposure ratio being based on a first amount of log entries associated with a first test version and a second amount of log entries associated with a second test version; and upon determining the exposure ratio is outside of the range, directing all requests from subsequent test users to a default test version.

公开(公告)号：US20220067762A1

公开(公告)日：2022-03-03

申请号：US17003705

申请日：2020-08-26

Applicant: Coupang Corp.

Inventor： Xiaowei GONG ,  Beibei Ye ,  Jun Ye ,  Chul Seo

IPC: G06Q30/02 ,  G06F17/18

Abstract: Computer-implemented systems and methods for predicting an optimal stop point during an experiment test are disclosed. A disclosed computer-implemented system comprises a memory storing instructions and at least one or more processors. The at least one or more processors may be configured to execute the instructions to obtain a total test time, obtain a minimum detectable effect trend data, determine an average minimum detectable effect change, determine a minimum detectable effect cumulative change threshold, determine a plurality of instantaneous minimum detectable effect changes, and determine a plurality of cumulative minimum detectable effect changes. Furthermore, the at least one or more processors may be configured to determine an optimal stop point time based on the average minimum detectable effect change, the plurality of instantaneous minimum detectable effect changes, and the minimum detectable effect cumulative change threshold to provide the optimal stop point time to a server to conclude the active test.

公开(公告)号：US11232018B1

公开(公告)日：2022-01-25

申请号：US17005834

申请日：2020-08-28

Applicant: COUPANG CORP.

Inventor： Ngoc-Lan Isabelle Phan ,  Jun Ye ,  Beibei Ye ,  Chul Seo

IPC: G06F11/00 ,  G06F11/36 ,  G06Q30/02 ,  G06F16/248 ,  G06F16/245

Abstract: A computerized method for controlled testing, comprising: providing a design specification for event data, the design specification including at least one of an event data source, an event data filter, and an event format; retrieving testing metrics; configuring testing events based on the design specification, by: retrieving testing events from the event data source specified in the design specification; filtering the retrieved testing events based in the design specification; and formatting the filtered testing events based on the design specification; generating analysis data by applying testing metrics to the configured event data; generating output data based on testing rules; and generating a user interface to display the output data.

公开(公告)号：US11032158B1

公开(公告)日：2021-06-08

申请号：US16951133

申请日：2020-11-18

Applicant: Coupang, Corp.

Inventor： Jun Ye ,  Chul Seo

IPC: H04L12/24 ,  G06F40/205

Abstract: A system for live analysis of testing logs. The system including a memory and a processor configured to execute the instructions to perform operations including receiving a plurality of log entries; processing the plurality of log entries; storing the processed plurality of log entries in a database having an inverted index; receiving a query from a user device; returning test data, metadata, and statistics related to the one or more log entries; displaying the test data, the metadata, and the statistics on a GUI; comparing an exposure ratio to a range, the exposure ratio being based on a first amount of log entries associated with a first test version and a second amount of log entries associated with a second test version; and upon determining the exposure ratio is outside of the range, directing all requests from subsequent test users to a default test version.

公开(公告)号：US11797423B2

公开(公告)日：2023-10-24

申请号：US17380152

申请日：2021-07-20

Applicant: COUPANG CORP.

Inventor： Ngoc-Lan Isabelle Phan ,  Jun Ye ,  Beibei Ye ,  Chul Seo

IPC: G06F11/00 ,  G06F11/36 ,  G06Q30/0201 ,  G06F16/248 ,  G06F16/245

CPC classification number: G06F11/3672 ,  G06F11/3664 ,  G06F11/3684 ,  G06F11/3688 ,  G06F16/245 ,  G06F16/248 ,  G06Q30/0201

Abstract: A computerized method for controlled testing, comprising: providing a design specification for event data, the design specification including at least one of an event data source, an event data filter, and an event format; retrieving testing metrics; configuring testing events based on the design specification, by: retrieving testing events from the event data source specified in the design specification; filtering the retrieved testing events based in the design specification; and formatting the filtered testing events based on the design specification; generating analysis data by applying testing metrics to the configured event data; generating output data based on testing rules; and generating a user interface to display the output data.