公开(公告)号：US09391861B1

公开(公告)日：2016-07-12

申请号：US13963986

申请日：2013-08-09

Applicant: Google Inc.

Inventor： Tyler William Odean ,  Marc-André Decoste ,  Jesse Doherty ,  Vadim Berman ,  Alexei Iourievich Svitkine ,  Steven Chi Truong ,  Ilya Mikhailovich Sherman

IPC: G06F15/173 ,  H04L12/26

CPC classification number: H04L67/306

Abstract: A computer-implemented method for comparing multiple client application variations, including, providing a copy of a client application to each of a plurality of client devices, wherein each copy of the client application comprises multiple application variations, receiving demographic information from one or more of the plurality of client devices and providing a list of available experiments to each of the plurality of client devices based on the demographic information, wherein each of the available experiments corresponds with one or more application variations. In certain implementations, the method further includes steps for receiving usage information for each copy of the client application associated with each of the plurality of client devices and comparing two or more application variations of the client application based on the usage information. Systems and computer-readable media are also provided.

Abstract translation: 一种用于比较多个客户端应用程序变体的计算机实现的方法，包括向客户端设备中的每一个提供客户端应用的副本，其中客户端应用的每个副本包括多个应用程序变体，从一个或多个 多个客户端设备，并且基于人口统计信息向多个客户端设备中的每一个提供可用实验的列表，其中每个可用实验对应于一个或多个应用变化。 在某些实现中，该方法还包括用于接收与多个客户端设备中的每一个相关联的客户端应用的每个副本的使用信息的步骤，并且基于使用信息来比较客户端应用的两个或多个应用变体。 还提供系统和计算机可读介质。

公开(公告)号：US08972770B1

公开(公告)日：2015-03-03

申请号：US13649952

申请日：2012-10-11

Applicant: Google Inc.

Inventor： Tyler William Odean ,  Ilya Mikhailovich Sherman

IPC: G06F11/00

CPC classification number: G06F11/3672

Abstract: Systems and methods for conducting a set of experiments on an application having a plurality of users are provided. Each experiment is identified by a respective experiment identification (ID) and is associated with a respective group of experimental variations of the application. A system includes an entropy source module that assigns a user group number to a designated user. The user group number is less than or equal to an entropy cap, which is less than an expected entropy, which represents a total number of different states of the experiments. The system includes an experiment module that determines a designated one of the experiments to be conducted on the application. The designated experiment is identified by a designated experiment ID and is associated with a designated group of experimental variations of the application. The experiment module assigns a designated variation to the designated user based on the user group number.

Abstract translation: 提供了用于在具有多个用户的应用上进行一组实验的系统和方法。 每个实验通过相应的实验标识（ID）标识，并与应用的各组实验变体相关联。 系统包括将用户组号分配给指定用户的熵源模块。 用户组号小于或等于熵帽，其小于预期熵，其表示实验的不同状态的总数。 该系统包括实验模块，该实验模块确定要在应用上进行的指定的一个实验。 指定的实验由指定的实验ID识别，并与应用的指定实验变体组相关联。 实验模块基于用户组号向指定的用户分配指定的变体。