-
公开(公告)号:US20200341880A1
公开(公告)日:2020-10-29
申请号:US16392892
申请日:2019-04-24
Applicant: Google LLC
Inventor: Mariana Sharp , Craig Chambers , Jelena Pjesivac-Grbovic , Kym Hines , Sunghwan Ihm , Danijela Mijailovic , Ben Walters
Abstract: Aspects of the disclosure relate to evaluating the rollout of a change to a data processing framework (DPF). A first set of jobs is selected out of a plurality of jobs. The first set of jobs is processed by the DPF with the proposed change enabled and the other jobs of the plurality of jobs by the DPF without the change enabled. A performance cost value is determined for each of the jobs in the first set of jobs processed by the DPF with the proposed change enabled and for each of the other jobs of the plurality of jobs processed by the DPF with the proposed change disabled. Based on the performance cost values of the jobs in the first set of jobs and the performance cost values of the other jobs of the plurality of jobs, a determination is made whether the proposed change is beneficial to the DPF.
-
公开(公告)号:US11726893B2
公开(公告)日:2023-08-15
申请号:US17465397
申请日:2021-09-02
Applicant: Google LLC
Inventor: Mariana Sharp , Craig D. Chambers , Jelena Pjesivac-Grbovic , Kym Hines , Sunghwan Ihm , Danijela Mijailovic , Ben Walters
CPC classification number: G06F11/3409 , G06F9/46 , G06F11/3006 , G06F9/44505
Abstract: Aspects of the disclosure relate to evaluating the rollout of a change to a data processing framework (DPF). A first set of jobs is selected out of a plurality of jobs. The first set of jobs is processed by the DPF with the proposed change enabled and the other jobs of the plurality of jobs by the DPF without the change enabled. A performance cost value is determined for each of the jobs in the first set of jobs processed by the DPF with the proposed change enabled and for each of the other jobs of the plurality of jobs processed by the DPF with the proposed change disabled. Based on the performance cost values of the jobs in the first set of jobs and the performance cost values of the other jobs of the plurality of jobs, a determination is made whether the proposed change is beneficial to the DPF.
-
3.
公开(公告)号:US20190171493A1
公开(公告)日:2019-06-06
申请号:US16146221
申请日:2018-09-28
Applicant: Google LLC
Inventor: Kenneth J. Goldman , Sunghwan Ihm , Xiaozhou Li , Jelena Pjesivac-Grbovic
Abstract: A manager agent access a grain hierarchy that represents a state for a computation that is to performed by a distributed processing system, wherein the grain hierarchy includes manager agent grains including metadata for manager agent processes that manage the performance of the computation by the distributed processing system, and worker agent grains including, for tasks to be performed by the distributed processing system, metadata for worker agents that each correspond to a subset of the plurality of data processors for performing the task. A manager agent performs processes defined by a manager agent grain to manage the computation by worker agents and storing, within the grain, metadata describing the manager agent process performed by the manager agent, and worker agents perform tasks assigned to the worker agents based on an assignment of a respective worker agent grain to the worker agent.
-
公开(公告)号:US20210397534A1
公开(公告)日:2021-12-23
申请号:US17465397
申请日:2021-09-02
Applicant: Google LLC
Inventor: Mariana Sharp , Craig D. Chambers , Jelena Pjesivac-Grbovic , Kym Hines , Sunghwan Ihm , Danijela Mijailovic , Ben Walters
Abstract: Aspects of the disclosure relate to evaluating the rollout of a change to a data processing framework (DPF). A first set of jobs is selected out of a plurality of jobs. The first set of jobs is processed by the DPF with the proposed change enabled and the other jobs of the plurality of jobs by the DPF without the change enabled. A performance cost value is determined for each of the jobs in the first set of jobs processed by the DPF with the proposed change enabled and for each of the other jobs of the plurality of jobs processed by the DPF with the proposed change disabled. Based on the performance cost values of the jobs in the first set of jobs and the performance cost values of the other jobs of the plurality of jobs, a determination is made whether the proposed change is beneficial to the DPF.
-
公开(公告)号:US11138087B2
公开(公告)日:2021-10-05
申请号:US16392892
申请日:2019-04-24
Applicant: Google LLC
Inventor: Mariana Sharp , Craig Chambers , Jelena Pjesivac-Grbovic , Kym Hines , Sunghwan Ihm , Danijela Mijailovic , Ben Walters
Abstract: Aspects of the disclosure relate to evaluating the rollout of a change to a data processing framework (DPF). A first set of jobs is selected out of a plurality of jobs. The first set of jobs is processed by the DPF with the proposed change enabled and the other jobs of the plurality of jobs by the DPF without the change enabled. A performance cost value is determined for each of the jobs in the first set of jobs processed by the DPF with the proposed change enabled and for each of the other jobs of the plurality of jobs processed by the DPF with the proposed change disabled. Based on the performance cost values of the jobs in the first set of jobs and the performance cost values of the other jobs of the plurality of jobs, a determination is made whether the proposed change is beneficial to the DPF.
-
公开(公告)号:US10754705B2
公开(公告)日:2020-08-25
申请号:US16146221
申请日:2018-09-28
Applicant: Google LLC
Inventor: Kenneth J. Goldman , Sunghwan Ihm , Xiaozhou Li , Jelena Pjesivac-Grbovic
Abstract: A manager agent access a grain hierarchy that represents a state for a computation that is to performed by a distributed processing system, wherein the grain hierarchy includes manager agent grains including metadata for manager agent processes that manage the performance of the computation by the distributed processing system, and worker agent grains including, for tasks to be performed by the distributed processing system, metadata for worker agents that each correspond to a subset of the plurality of data processors for performing the task. A manager agent performs processes defined by a manager agent grain to manage the computation by worker agents and storing, within the grain, metadata describing the manager agent process performed by the manager agent, and worker agents perform tasks assigned to the worker agents based on an assignment of a respective worker agent grain to the worker agent.
-
7.
公开(公告)号:US10089154B2
公开(公告)日:2018-10-02
申请号:US15240785
申请日:2016-08-18
Applicant: Google LLC
Inventor: Kenneth J. Goldman , Sunghwan Ihm , Xiaozhou Li , Jelena Pjesivac-Grbovic
Abstract: A manager agent access a grain hierarchy that represents a state for a computation that is to performed by a distributed processing system, wherein the grain hierarchy includes manager agent grains including metadata for manager agent processes that manage the performance of the computation by the distributed processing system, and worker agent grains including, for tasks to be performed by the distributed processing system, metadata for worker agents that each correspond to a subset of the plurality of data processors for performing the task. A manager agent performs processes defined by a manager agent grain to manage the computation by worker agents and storing, within the grain, metadata describing the manager agent process performed by the manager agent, and worker agents perform tasks assigned to the worker agents based on an assignment of a respective worker agent grain to the worker agent.
-
8.
公开(公告)号:US11620168B2
公开(公告)日:2023-04-04
申请号:US16987636
申请日:2020-08-07
Applicant: Google LLC
Inventor: Kenneth Jerome Goldman , Sunghwan Ihm , Xiaozhou Li , Jelena Pjesivac-Grbovic
Abstract: A manager agent access a grain hierarchy that represents a state for a computation that is to performed by a distributed processing system, wherein the grain hierarchy includes manager agent grains including metadata for manager agent processes that manage the performance of the computation by the distributed processing system, and worker agent grains including, for tasks to be performed by the distributed processing system, metadata for worker agents that each correspond to a subset of the plurality of data processors for performing the task. A manager agent performs processes defined by a manager agent grain to manage the computation by worker agents and storing, within the grain, metadata describing the manager agent process performed by the manager agent, and worker agents perform tasks assigned to the worker agents based on an assignment of a respective worker agent grain to the worker agent.
-
9.
公开(公告)号:US20210042166A1
公开(公告)日:2021-02-11
申请号:US16987636
申请日:2020-08-07
Applicant: Google LLC
Inventor: Kenneth Jerome Goldman , Sunghwan Ihm , Xiaozhou Li , Jelena Pjesivac-Grbovic
Abstract: A manager agent access a grain hierarchy that represents a state for a computation that is to performed by a distributed processing system, wherein the grain hierarchy includes manager agent grains including metadata for manager agent processes that manage the performance of the computation by the distributed processing system, and worker agent grains including, for tasks to be performed by the distributed processing system, metadata for worker agents that each correspond to a subset of the plurality of data processors for performing the task. A manager agent performs processes defined by a manager agent grain to manage the computation by worker agents and storing, within the grain, metadata describing the manager agent process performed by the manager agent, and worker agents perform tasks assigned to the worker agents based on an assignment of a respective worker agent grain to the worker agent.
-
-
-
-
-
-
-
-