-
公开(公告)号：US09596298B1
公开(公告)日：2017-03-14
申请号：US14144826
申请日：2013-12-31
Applicant: Google Inc.
Inventor： Kaya Bekiroglu , Vyacheslav Alekseyevich Chernyak
IPC: G06F15/173 , H04L29/08 , G06F9/50 , G06F9/52
CPC classification number: H04L67/1002 , G06F9/505 , G06F9/5083 , G06F9/522 , H04L67/1008
Abstract: Methods, systems, and apparatus are described for load balancing in a distributed processing system. In one aspect, a method includes monitoring, for each data processor in a distributed processing system, a rate of cursor progress for the data processor based on timestamps of data units that have been processed, wherein the rate of cursor progress for each data processor specifies a rate of change of an oldest timestamp of an oldest data unit being processed by the data processor; determining a progress threshold for the distributed processing system based on the respective rates of cursor progress of the data processors; determining, based on a first rate of cursor progress for a first data processor, that the first rate of cursor progress does not meet the progress threshold; and in response to the determination, performing one or more load balancing operations on the distributed processing system.
Abstract translation: 描述了分布式处理系统中的负载平衡的方法，系统和装置。 在一个方面，一种方法包括基于已处理的数据单元的时间戳来监视分布式处理系统中的每个数据处理器对数据处理器的光标进度的速率，其中每个数据处理器的光标进度速率指定 由数据处理器处理的最早数据单元的最早时间戳的变化率; 基于数据处理器的光标进度的相应速率来确定分布式处理系统的进度阈值; 基于第一数据处理器的光标进度的第一速率确定所述第一光标进度速率不满足所述进度阈值; 并且响应于所述确定，对所述分布式处理系统执行一个或多个负载平衡操作。
-
公开(公告)号：US09767217B1
公开(公告)日：2017-09-19
申请号：US14288456
申请日：2014-05-28
Applicant: Google Inc.
Inventor： Matthew Harold Austern , Reuven Lax , Vyacheslav Alekseyevich Chernyak
CPC classification number: G06F17/30569 , G06F9/5038 , G06F17/30312 , G06F17/30424
Abstract: Methods, systems, and apparatus are described for performing streaming graph computations in a distributed processing system. In one aspect, a method includes receiving, during a first superstep, streaming graph input that comprises continuous input of first graph updates for updating the state of an existing graph, the existing graph comprising a plurality of vertices and edges, and each first graph update specifying a vertex and a message; scheduling, during the first superstep, the first graph updates for processing during a second superstep; providing, during the first superstep, the first graph updates as input to a graph computation for the existing graph; determining that the graph computation has completed processing for the first superstep, and in response: initiating the second superstep; and processing, during the second superstep, each first graph update; and providing, during the second superstep, graph output based at least in part on the first graph updates.
-
公开(公告)号：US09621636B1
公开(公告)日：2017-04-11
申请号：US14023208
申请日：2013-09-10
Applicant: Google Inc.
Inventor： Samuel Green McVeety , Vyacheslav Alekseyevich Chernyak
CPC classification number: H04L67/10 , G06F9/5083 , H04L67/32
Abstract: Methods, systems, and apparatus are described for throttling a distributed processing system. In one aspect, a method includes identifying records being processed by a distributed processing system that performs agent processes, each of the records including a corresponding timestamp; determining, based on timestamps of the records that have been processed by a first agent process, a first agent progress; identifying a dependent agent process performed by the distributed processing system, wherein the dependent agent process processes only records that have been processed by the first agent process; determining, based on timestamps of records that have been processed by the dependent agent process, a dependent agent progress; and throttling performance of the first process based on the first agent progress and the dependent agent progress.
-
-
Search Results
3
records
(took 0.012 seconds)
