公开(公告)号：US20130254771A1

公开(公告)日：2013-09-26

申请号：US13726958

申请日：2012-12-26

Applicant: GOOGLE INC.

Inventor： Eldar A. Musayev ,  Nikunj Bhagat ,  Ian Porteous ,  Laramie J. Leavitt ,  Matthew Nichols

IPC: G06F9/46

CPC classification number: G06F9/466 ,  G06F9/4843

Abstract: Methods, systems and apparatus are described herein that include processing a data stream as a sequence of batch jobs during collection of data in the data stream. Processing of successive batch jobs in the sequence includes creating a particular batch job upon completion of processing of a preceding batch job in the sequence. The particular batch job has a batch size that depends upon an amount of data in the data stream that has been collected since creation of the preceding batch job in the sequence, such that the batch size of the particular batch job self-adjusts to data rate changes in the data stream. The particular batch job is then processed to produce resulting data, where processing efficiency and processing time for the particular batch increase with the batch size.

Abstract translation: 本文描述的方法，系统和装置包括在数据流中的数据收集期间将数据流处理为批处理作业的序列。 处理序列中的连续批处理作业包括在序列中处理先前的批处理作业之后创建特定批处理作业。 特定批处理作业具有取决于从序列中创建上一批次作业以来收集的数据流中的数据量的批量大小，使得特定批处理作业的批量大小自适应到数据速率 数据流中的变化。 然后处理特定批处理作业以生成结果数据，其中特定批处理的处理效率和处理时间随批处理大小而增加。