公开(公告)号：US20220114079A1

公开(公告)日：2022-04-14

申请号：US17232229

申请日：2021-04-16

Applicant: Coupang Corp.

Inventor： Ngoc-Lan Isabelle Phan ,  Beibei Ye ,  Chul Seo

IPC: G06F11/36

Abstract: Disclosed herein are systems and methods for a system for distributing user requests. The system may comprise a memory storing instructions and at least one processor configured to execute instructions to perform operations. These operations may comprise receiving from a local network device, without requesting, a data randomization scheme determining a test dataset from among a plurality of test datasets for deployment at a client device; based on the received data randomization scheme, determining test participation data for transmitting to a plurality of client devices; transmitting the test participation data to the plurality of client devices; transmitting, to the plurality of client devices, datasets configured for deployment by the plurality of client devices according to the test participation data; and receiving, from at least one of the plurality of client devices, interaction data associated with one of the datasets deployed by the at least one client device according to the test participation data.

公开(公告)号：US11797423B2

公开(公告)日：2023-10-24

申请号：US17380152

申请日：2021-07-20

Applicant: COUPANG CORP.

Inventor： Ngoc-Lan Isabelle Phan ,  Jun Ye ,  Beibei Ye ,  Chul Seo

IPC: G06F11/00 ,  G06F11/36 ,  G06Q30/0201 ,  G06F16/248 ,  G06F16/245

CPC classification number: G06F11/3672 ,  G06F11/3664 ,  G06F11/3684 ,  G06F11/3688 ,  G06F16/245 ,  G06F16/248 ,  G06Q30/0201

Abstract: A computerized method for controlled testing, comprising: providing a design specification for event data, the design specification including at least one of an event data source, an event data filter, and an event format; retrieving testing metrics; configuring testing events based on the design specification, by: retrieving testing events from the event data source specified in the design specification; filtering the retrieved testing events based in the design specification; and formatting the filtered testing events based on the design specification; generating analysis data by applying testing metrics to the configured event data; generating output data based on testing rules; and generating a user interface to display the output data.

公开(公告)号：US20220066913A1

公开(公告)日：2022-03-03

申请号：US17380152

申请日：2021-07-20

Applicant: Coupang Corp.

Inventor： Ngoc-Lan Isabelle Phan ,  Jun Ye ,  Beibel Ye ,  Chul Seo

IPC: G06F11/36 ,  G06Q30/02 ,  G06F16/248 ,  G06F16/245

Abstract: A computerized method for controlled testing, comprising: providing a design specification for event data, the design specification including at least one of an event data source, an event data filter, and an event format; retrieving testing metrics; configuring testing events based on the design specification, by: retrieving testing events from the event data source specified in the design specification; filtering the retrieved testing events based in the design specification; and formatting the filtered testing events based on the design specification; generating analysis data by applying testing metrics to the configured event data; generating output data based on testing rules; and generating a user interface to display the output data.

公开(公告)号：US11620210B2

公开(公告)日：2023-04-04

申请号：US17232229

申请日：2021-04-16

Applicant: Coupang Corp.

Inventor： Ngoc-Lan Isabelle Phan ,  Beibei Ye ,  Chul Seo

IPC: G06F11/00 ,  G06F11/36

Abstract: Disclosed herein are systems and methods for a system for distributing user requests. The system may comprise a memory storing instructions and at least one processor configured to execute instructions to perform operations. These operations may comprise receiving from a local network device, without requesting, a data randomization scheme determining a test dataset from among a plurality of test datasets for deployment at a client device; based on the received data randomization scheme, determining test participation data for transmitting to a plurality of client devices; transmitting the test participation data to the plurality of client devices; transmitting, to the plurality of client devices, datasets configured for deployment by the plurality of client devices according to the test participation data; and receiving, from at least one of the plurality of client devices, interaction data associated with one of the datasets deployed by the at least one client device according to the test participation data.

公开(公告)号：US11010281B1

公开(公告)日：2021-05-18

申请号：US17068529

申请日：2020-10-12

Applicant: Coupang Corp.

Inventor： Ngoc-Lan Isabelle Phan ,  Beibei Ye ,  Chul Seo

IPC: G06F11/00 ,  G06F11/36

Abstract: Disclosed herein are systems and methods for a system for distributing user requests. The system may comprise a memory storing instructions and at least one processor configured to execute instructions to perform operations. These operations may comprise receiving from a local network device, without requesting, a data randomization scheme determining a test dataset from among a plurality of test datasets for deployment at a client device; based on the received data randomization scheme, determining test participation data for transmitting to a plurality of client devices; transmitting the test participation data to the plurality of client devices; transmitting, to the plurality of client devices, datasets configured for deployment by the plurality of client devices according to the test participation data; and receiving, from at least one of the plurality of client devices, interaction data associated with one of the datasets deployed by the at least one client device according to the test participation data.

公开(公告)号：US20220067762A1

公开(公告)日：2022-03-03

申请号：US17003705

申请日：2020-08-26

Applicant: Coupang Corp.

Inventor： Xiaowei GONG ,  Beibei Ye ,  Jun Ye ,  Chul Seo

IPC: G06Q30/02 ,  G06F17/18

Abstract: Computer-implemented systems and methods for predicting an optimal stop point during an experiment test are disclosed. A disclosed computer-implemented system comprises a memory storing instructions and at least one or more processors. The at least one or more processors may be configured to execute the instructions to obtain a total test time, obtain a minimum detectable effect trend data, determine an average minimum detectable effect change, determine a minimum detectable effect cumulative change threshold, determine a plurality of instantaneous minimum detectable effect changes, and determine a plurality of cumulative minimum detectable effect changes. Furthermore, the at least one or more processors may be configured to determine an optimal stop point time based on the average minimum detectable effect change, the plurality of instantaneous minimum detectable effect changes, and the minimum detectable effect cumulative change threshold to provide the optimal stop point time to a server to conclude the active test.

公开(公告)号：US11232018B1

公开(公告)日：2022-01-25

申请号：US17005834

申请日：2020-08-28

Applicant: COUPANG CORP.

Inventor： Ngoc-Lan Isabelle Phan ,  Jun Ye ,  Beibei Ye ,  Chul Seo

IPC: G06F11/00 ,  G06F11/36 ,  G06Q30/02 ,  G06F16/248 ,  G06F16/245

Abstract: A computerized method for controlled testing, comprising: providing a design specification for event data, the design specification including at least one of an event data source, an event data filter, and an event format; retrieving testing metrics; configuring testing events based on the design specification, by: retrieving testing events from the event data source specified in the design specification; filtering the retrieved testing events based in the design specification; and formatting the filtered testing events based on the design specification; generating analysis data by applying testing metrics to the configured event data; generating output data based on testing rules; and generating a user interface to display the output data.

公开(公告)号：US11032158B1

公开(公告)日：2021-06-08

申请号：US16951133

申请日：2020-11-18

Applicant: Coupang, Corp.

Inventor： Jun Ye ,  Chul Seo

IPC: H04L12/24 ,  G06F40/205

Abstract: A system for live analysis of testing logs. The system including a memory and a processor configured to execute the instructions to perform operations including receiving a plurality of log entries; processing the plurality of log entries; storing the processed plurality of log entries in a database having an inverted index; receiving a query from a user device; returning test data, metadata, and statistics related to the one or more log entries; displaying the test data, the metadata, and the statistics on a GUI; comparing an exposure ratio to a range, the exposure ratio being based on a first amount of log entries associated with a first test version and a second amount of log entries associated with a second test version; and upon determining the exposure ratio is outside of the range, directing all requests from subsequent test users to a default test version.