公开(公告)号：US20170337241A1

公开(公告)日：2017-11-23

申请号：US15597302

申请日：2017-05-17

Applicant: Ab Initio Technology LLC

Inventor： Jeffrey Newbern ,  Craig W. Stanfill

IPC: G06F17/30 ,  H04L12/24

CPC classification number: G06F16/2365 ,  G06F9/4494 ,  G06F9/5061 ,  G06F9/5066 ,  G06F16/24568 ,  G06F16/27 ,  G06F16/278 ,  G06F16/285 ,  H04L41/0803

Abstract: Distributed processing of a data collection includes receiving information for configuring a distributed processing system. A first configuration of components is formed including sources of data elements and workers configured to process data elements, distributed among computing resources. Each data element includes a partition value that identifies a subset of the workers according to a partition rule. Data elements are accepted from the sources for a first part of the data collection in a first processing epoch and the data elements are routed through the first configuration. After accepting a first part of the data collection, change of configuration is initiated to a second configuration. A succession of two or more transitions between configurations of components is performed to a succession of modified configurations, a last of which corresponds to the second configuration. Further data elements are accepted from sources of the second configuration in a second processing epoch.

公开(公告)号：US10606827B2

公开(公告)日：2020-03-31

申请号：US15597302

申请日：2017-05-17

Applicant: Ab Initio Technology LLC

Inventor： Jeffrey Newbern ,  Craig W. Stanfill

IPC: G06F16/23 ,  G06F9/448 ,  G06F16/27 ,  G06F16/28 ,  G06F16/2455 ,  G06F9/50 ,  H04L12/24

Abstract: Distributed processing of a data collection includes receiving information for configuring a distributed processing system. A first configuration of components is formed including sources of data elements and workers configured to process data elements, distributed among computing resources. Each data element includes a partition value that identifies a subset of the workers according to a partition rule. Data elements are accepted from the sources for a first part of the data collection in a first processing epoch and the data elements are routed through the first configuration. After accepting a first part of the data collection, change of configuration is initiated to a second configuration. A succession of two or more transitions between configurations of components is performed to a succession of modified configurations, a last of which corresponds to the second configuration. Further data elements are accepted from sources of the second configuration in a second processing epoch.

公开(公告)号：US20210049146A1

公开(公告)日：2021-02-18

申请号：US16832111

申请日：2020-03-27

Applicant: Ab Initio Technology LLC

Inventor： Jeffrey Newbern ,  Craig W. Stanfill

IPC: G06F16/23 ,  G06F9/448 ,  G06F16/27 ,  G06F16/28 ,  G06F16/2455 ,  G06F9/50 ,  H04L12/24

Abstract: Distributed processing of a data collection includes receiving information for configuring a distributed processing system. A first configuration of components is formed including sources of data elements and workers configured to process data elements, distributed among computing resources. Each data element includes a partition value that identifies a subset of the workers according to a partition rule. Data elements are accepted from the sources for a first part of the data collection in a first processing epoch and the data elements are routed through the first configuration. After accepting a first part of the data collection, change of configuration is initiated to a second configuration. A succession of two or more transitions between configurations of components is performed to a succession of modified configurations, a last of which corresponds to the second configuration. Further data elements are accepted from sources of the second configuration in a second processing epoch.

公开(公告)号：US20170177414A1

公开(公告)日：2017-06-22

申请号：US15368750

申请日：2016-12-05

Applicant: Ab Initio Technology LLC

Inventor： Jeffrey Newbern ,  Craig W. Stanfill

IPC: G06F9/50

Abstract: Work units are received into, and processed from, holding queues. A partition, associated with a unique holding queue, is assigned to each work unit stored in a given holding queue, according to a partitioning rule. Work units are separated from each holding queue into one of two or more other queues stored in the same storage as that holding queue. A ready queue (relocation queue) is for each work unit whose assigned partition is (is not) associated with the holding queue in which it is stored. Each work unit in each relocation queue is relocated to the holding queue associated with the partition assigned to that work unit. The work units are repartitioned in response to a change in the partitioning rule to a new partitioning rule before the holding queues are empty.