-
公开(公告)号:US20220391257A1
公开(公告)日:2022-12-08
申请号:US17831978
申请日:2022-06-03
Applicant: Ab Initio Technology LLC
Inventor: Stephen J. Schmidt , David Ricardo , Craig W. Stanfill , Jacky Cheuk Kei Chu , Ephraim Meriwether Vishniac
IPC: G06F9/50
Abstract: A method for performing a distributed computation on a computing system using computational resources dynamically allocated using a computational resource manager includes storing information specifying quantities of computational resources associated with respective ones of a number of program portions of the program, where the program portions perform successive transformations of data and each program portion uses computational resources granted by the computational resource manager enabling computation associated with that program portion to be performed in the computing system, requesting a first quantity of computational resources associated with a first program portion of the number of program portions from the computational resource manager, receiving a second quantity of computational resources from the computational resource manager, less than the requested first quantity of computational resources, performing computation associated with the first portion of the program using the second quantity of computational resources, while performing the computation associated with the first portion of the program using the second quantity of computational resources, receiving an additional quantity of computational resources from the computational resource manager, and performing an additional computation associated with the first portion of the program using the additional quantity of computational resources while performing the computation associated with the first portion using the second quantity of computational resources.
-
公开(公告)号:US11301445B2
公开(公告)日:2022-04-12
申请号:US16701193
申请日:2019-12-03
Applicant: Ab Initio Technology LLC
Inventor: Craig W. Stanfill , Richard Shapiro , Stephen A. Kukolich
Abstract: A graph-based program specification includes: a plurality of components, each corresponding to a processing task and including one or more ports for sending or receiving one or more data elements; and one or more links, each connecting an output port of an upstream component of the plurality of components to an input port of a downstream component of the plurality of components. Prepared code is generated representing subsets of the plurality of components, including: identifying a plurality of subset boundaries between components in different subsets based at least in part on characteristics of linked components; forming the subsets based on the identified subset boundaries; and generating prepared code for each formed subset that when used for execution by a runtime system causes processing tasks corresponding to the components in that formed subset to be performed according to information embedded in the prepared code for that formed subset.
-
公开(公告)号:US11269918B2
公开(公告)日:2022-03-08
申请号:US16175454
申请日:2018-10-30
Applicant: Ab Initio Technology LLC
Inventor: Bryan Phil Douros , Craig W. Stanfill , Joseph Skeffington Wholey, III
Abstract: A method for managing a distributed data processing system, the method implementing counters to track durability states of data units in the distributed data processing system, wherein the counters are used to manage processing of the data units in the distributed data processing system.
-
公开(公告)号:US10896025B2
公开(公告)日:2021-01-19
申请号:US16214240
申请日:2018-12-10
Applicant: Ab Initio Technology LLC
Inventor: Craig W. Stanfill , Adam Weiss , Andrew F. Roberts , Stephen A. Kukolich
Abstract: User input is received specifying components of a graph-based program specification. User input is received specifying links, at least some connecting an output port of an upstream component to an input port of a downstream component. The graph-based program specification is processed to identify one or more subsets of the components, including: identifying one or more subset entry points and one or more subset exit points that occur between components in different subsets based at least in part on data processing characteristics of linked components, and forming the subsets based on the identified subset entry points and exit points. A visual representation of the formed subsets is rendered within a user interface. Prepared code is generated for each formed subset that when used for execution by a runtime system causes processing tasks corresponding to the components in each formed subset to be performed.
-
公开(公告)号:US10579753B2
公开(公告)日:2020-03-03
申请号:US15376129
申请日:2016-12-12
Applicant: Ab Initio Technology LLC
Inventor: Joel Gould , Scott Studer , Craig W. Stanfill
IPC: G06G7/62 , G06F17/50 , G06F16/2455 , G06F11/36
Abstract: A method implemented by a data processing system for processing data items of a stream of data items, including: accessing a specification that represents the executable logic, wherein a state of the specification for a particular value of the key specifies one or more portions of the executable logic that are executable in that state; receiving, over an input device or port, data items of a stream of data; for a first one of the data items of the stream, identifying a first state of the specification for a value of the key associated with that first one of the data items; processing, by the data processing system, the first one of the data items according to one or more portions of executable logic that are represented in the specification as being associated with the first state.
-
公开(公告)号:US10558493B2
公开(公告)日:2020-02-11
申请号:US16175133
申请日:2018-10-30
Applicant: Ab Initio Technology LLC
Inventor: Craig W. Stanfill , Joseph Skeffington Wholey, III
Abstract: A method for processing state update requests in a distributed data processing system with a number of processing nodes includes maintaining a number of counters including a working counter indicating a current time interval, a replication counter indicating a time interval for which all requests associated with that time interval are replicated at multiple processing nodes of the number of processing nodes, and a persistence counter indicating a time interval of the number of time intervals for which all requests associated with that time interval are stored in persistent storage. The counters are used to manage processing of the state update requests.
-
公开(公告)号:US20190354348A1
公开(公告)日:2019-11-21
申请号:US16214240
申请日:2018-12-10
Applicant: Ab Initio Technology LLC
Inventor: Craig W. Stanfill , Adam Weiss , Andrew F. Roberts , Stephen A. Kukolich
IPC: G06F8/10 , G06F8/34 , G06F9/448 , G06F3/0482
Abstract: User input is received specifying components of a graph-based program specification. User input is received specifying links, at least some connecting an output port of an upstream component to an input port of a downstream component. The graph-based program specification is processed to identify one or more subsets of the components, including: identifying one or more subset entry points and one or more subset exit points that occur between components in different subsets based at least in part on data processing characteristics of linked components, and forming the subsets based on the identified subset entry points and exit points. A visual representation of the formed subsets is rendered within a user interface. Prepared code is generated for each formed subset that when used for execution by a runtime system causes processing tasks corresponding to the components in each formed subset to be performed.
-
公开(公告)号:US10423395B2
公开(公告)日:2019-09-24
申请号:US16042205
申请日:2018-07-23
Applicant: Ab Initio Technology LLC
Inventor: Craig W. Stanfill , Richard Shapiro
Abstract: A received graph-based program specification includes: a plurality of components, each corresponding to at least one operation; and a plurality of directed links each connecting an upstream component to a downstream component. Processed code is generated representing one or more groups of operations by: identifying a possible level of concurrency in a first group of operations based at least in part on a topology of the graph, such that multiple operations in the first group are not prevented by the topology of the graph from executing concurrently; analyzing at least some of the operations in the first group to determine runtime characteristics associated with the analyzed operations; and generating processed code for executing the operations, where the processed code enforces a reduced level of concurrency in the first group, lower than the identified possible level of concurrency, based at least in part on the determined runtime characteristics.
-
公开(公告)号:US10191863B2
公开(公告)日:2019-01-29
申请号:US14658440
申请日:2015-03-16
Applicant: Ab Initio Technology LLC
Inventor: Jed Roberts , Craig W. Stanfill , Scott Studer
Abstract: One or more mappings each define a correspondence between input attributes of an input entity and output attributes of an output entity, where the input out output entities each include one or more key attributes identified as part of a unique key. Computing result information, displayed in a user interface, includes: processing instances of a first input entity to generate instances of a first output entity; determining one or more mapped input attributes of the first input entity that correspond to each of the key attributes of the first output entity; generating the instances of the first output entity based on the determined one or more mapped input attributes; computing a total number of instances of the first input entity that were processed; and computing a total number of instances of the first output entity that were generated.
-
公开(公告)号:US20180011861A1
公开(公告)日:2018-01-11
申请号:US15709786
申请日:2017-09-20
Applicant: Ab Initio Technology LLC
Inventor: Ephraim Meriwether Vishniac , Craig W. Stanfill
CPC classification number: G06F16/10 , G06F3/0608 , G06F3/0643 , G06F3/0676
Abstract: A method includes determining a length of a file and storing the length of the file in a first memory location. An endpoint of a last complete record within the file is determined and the endpoint is stored in a second memory location. The length of the file stored in the first memory location is compared to a current length of the file, and a data structure associated with the file is updated beginning at the endpoint if the current length of the file exceeds the length of the file stored in the first memory location.
-
-
-
-
-
-
-
-
-