公开(公告)号：US10089087B2

公开(公告)日：2018-10-02

申请号：US14843162

申请日：2015-09-02

Applicant: Ab Initio Technology LLC

Inventor： Craig W. Stanfill ,  Richard Shapiro ,  Adam Weiss ,  Andrew F. Roberts ,  Joseph Skeffington Wholey, III ,  Joel Gould ,  Stephen A. Kukolich

IPC: G06F8/41 ,  G06F13/36 ,  G06F9/445 ,  G06F9/50 ,  G06F9/448 ,  G06F8/34

Abstract: A graph-based program specification includes components corresponding to tasks and directed links between ports of the components, including: a first type of link configuration between ports of linked components, corresponding to transfer of control or transfer of a single data element, and a second type of link configuration between ports of linked components, corresponding to transfer of multiple data elements. A compiler generates a target program specification including control code representing at least one control graph including graph nodes representing the components, where at least two are connected based on links of the first type. A computing node initiates execution of the target program specification, and manages computing resources for links of the second type, the computing resources including at least one of: (1) a buffer for storing data elements provided by an output port, or (2) a buffer for storing data elements provided to an input port.

公开(公告)号：US20170371677A1

公开(公告)日：2017-12-28

申请号：US15685339

申请日：2017-08-24

Applicant: Ab Initio Technology LLC

Inventor： Craig W. Stanfill ,  Richard Shapiro ,  Stephen A. Kukolich ,  Joseph Skeffington Wholey, III

IPC: G06F9/44 ,  G06F9/46 ,  G06F9/48

CPC classification number: G06F9/44 ,  G06F8/34 ,  G06F9/448 ,  G06F9/4482 ,  G06F9/4494 ,  G06F9/46 ,  G06F9/465 ,  G06F9/466 ,  G06F9/4843 ,  G06F9/4881

Abstract: A graph-based program specification includes components, at least one having at least one input port for receiving a collection of data elements, or at least one collection type output port for providing a collection of data elements. Executing a program specified by the graph-based program specification at a computing node, includes: receiving data elements of a first collection into a first storage in a first order via a link connected to a collection type output port of a first component and an input port of a second component, and invoking a plurality of instances of a task corresponding to the second component to process data elements of the first collection, including retrieving the data elements from the first storage in a second order, without blocking invocation of any of the instances until after any particular instance completes processing one or more data elements.

公开(公告)号：US09830343B2

公开(公告)日：2017-11-28

申请号：US14842985

申请日：2015-09-02

Applicant: Ab Initio Technology LLC

Inventor： Craig W. Stanfill ,  Richard Shapiro ,  Stephen A. Kukolich

IPC: G06F9/44 ,  G06F9/45 ,  G06F9/50 ,  G06F17/30

CPC classification number: G06F17/30312 ,  G06F8/41 ,  G06F8/433 ,  G06F8/456 ,  G06F9/4494 ,  G06F9/5066 ,  G06F17/30958

Abstract: A graph-based program specification includes: a plurality of components, each corresponding to a processing task and including one or more ports, including scalar data ports for sending or receiving a single data element and collection data ports for sending or receiving a collection of multiple data elements; and one or more links, each connecting an output port of an upstream component to an input port of a downstream component. Prepared code is generated representing subsets of the plurality of components, including: identifying one or more subset boundaries, including identifying one or more links connecting a collection data port of a component to a scalar data port of a component; forming the subsets based on the identified subset boundaries; and generating prepared code for each formed subset that when used for execution by a runtime system causes processing tasks corresponding to the components in each formed subset to be performed.

公开(公告)号：US09753751B2

公开(公告)日：2017-09-05

申请号：US14520588

申请日：2014-10-22

Applicant: Ab Initio Technology LLC

Inventor： Matthew Darcy Atterbury ,  H. Mark Bromley ,  Wayne Mesard ,  Arkadi Popov ,  Stephen Schmidt ,  Craig W. Stanfill ,  Joseph Skeffington Wholey

IPC: G06F9/46 ,  G06F9/445 ,  G06F9/44

CPC classification number: G06F9/44521 ,  G06F9/44536 ,  G06F9/4494

Abstract: Processing data includes: receiving units of work that each include one or more work elements, and processing a first unit of work using a first compiled dataflow graph (160) loaded into a data processing system (100) in response to receiving the first unit of work. The processing includes: analysis to determine a characteristic of the first unit of work; identifying one or more compiled dataflow graphs from graphs stored in a data storage system (107) that include at least some that were compiled for processing a unit of work having the determined characteristic; loading one of the identified compiled dataflow graphs into the data processing system (100) as the first compiled dataflow graph (160); and generating one or more output work elements from at least one work element in the first unit of work.

公开(公告)号：US20160070729A1

公开(公告)日：2016-03-10

申请号：US14843120

申请日：2015-09-02

Applicant: Ab Initio Technology LLC

Inventor： Craig W. Stanfill ,  Richard Shapiro ,  Stephen A. Kukolich

IPC: G06F17/30

CPC classification number: G06F16/22 ,  G06F8/41 ,  G06F8/433 ,  G06F8/456 ,  G06F9/4494 ,  G06F9/5066 ,  G06F16/9024

Abstract: A graph-based program specification includes: a plurality of components, each corresponding to a processing task and including one or more ports for sending or receiving one or more data elements; and one or more links, each connecting an output port of an upstream component of the plurality of components to an input port of a downstream component of the plurality of components. Prepared code is generated representing subsets of the plurality of components, including: identifying a plurality of subset boundaries between components in different subsets based at least in part on characteristics of linked components; forming the subsets based on the identified subset boundaries; and generating prepared code for each formed subset that when used for execution by a runtime system causes processing tasks corresponding to the components in that formed subset to be performed according to information embedded in the prepared code for that formed subset.

Abstract translation: 基于图形的程序规范包括：多个组件，每个组件对应于处理任务，并且包括用于发送或接收一个或多个数据单元的一个或多个端口; 以及一个或多个链路，每个链路将多个部件的上游部件的输出端口连接到多个部件的下游部件的输入端口。 生成表示多个组件的子集的准备代码，包括：至少部分地基于链接的组件的特征来识别不同子集中的组件之间的多个子集边界; 基于所识别的子集边界形成子集; 并且为每个形成的子集生成准备的代码，当用于运行时系统的执行时，会根据嵌入在所形成的子集的准备代码中的信息来执行与形成的子集中的组件相对应的处理任务。

公开(公告)号：US20160062804A1

公开(公告)日：2016-03-03

申请号：US14843026

申请日：2015-09-02

Applicant: Ab Initio Technology LLC

Inventor： Craig W. Stanfill ,  Richard Shapiro

IPC: G06F9/50 ,  G06F9/45

CPC classification number: G06F9/5027 ,  G06F8/41 ,  G06F8/447 ,  G06F9/4494 ,  G06F9/4843

Abstract: Information representative of a graph-based program specification has components, and directed links between ports of said components, defining a dependency between said components. A directed link exists between a port of a first component and a port of a second component. The first component specifies first-component execution code that when compiled enables execution of a first task. The second component specifies second-component execution code that when compiled enables execution of a second task. Compiling the graph-based program specification includes grafting first control code to said first-component execution code, which changes a state of said second component to a pending state, an active state, or a suppressed state. Based on said state, said first control code causes at least one of: invoking said second component if said state changes from pending to active, or suppressing said second component if said state changes from pending to suppressed.

Abstract translation: 代表基于图表的程序规范的信息具有组件和所述组件的端口之间的定向链接，从而定义所述组件之间的依赖关系。 在第一组件的端口和第二组件的端口之间存在有向链接。 第一个组件指定第一个组件执行代码，当编译启用执行第一个任务时。 第二个组件指定第二个组件执行代码，当编译启用执行第二个任务时。 编译基于图形的程序规范包括将第一控制代码移植到所述第一组件执行代码，所述第一组件执行代码将所述第二组件的状态改变为待处理状态，活动状态或抑制状态。 基于所述状态，所述第一控制码导致以下中的至少一个：如果所述状态从挂起状态改变为活动状态则调用所述第二分量，或者如果所述状态从等待改变为抑制则抑制所述第二分量。

公开(公告)号：US20160062800A1

公开(公告)日：2016-03-03

申请号：US14842934

申请日：2015-09-02

Applicant: Ab Initio Technology LLC

Inventor： Craig W. Stanfill ,  Richard Shapiro ,  Adam Weiss ,  Andrew F. Roberts ,  Joseph Skeffington Wholey, III ,  Joel Gould

IPC: G06F9/50

CPC classification number: G06F9/5038 ,  G06F2209/506

Abstract: Information representative of a graph-based program specification has a plurality of components, each of which corresponds to a task, and directed links between ports of said components. A program corresponding to said graph-based program specification is executed. A first component includes a first data port, a first control port, and a second control port. Said first data port is configured to receive data to be processed by a first task corresponding to said first component, or configured to provide data that was processed by said first task corresponding to said first component. Executing a program corresponding to said graph-based program specification includes: receiving said first control information at said first control port, in response to receiving said first control information, determining whether or not to invoke said first task, and after receiving said first control information, providing said second control information from said second control port.

Abstract translation: 代表基于图表的程序规范的信息具有多个组件，每个组件对应于任务，并且指向所述组件的端口之间的链接。 执行与所述基于图形的程序规范相对应的程序。 第一组件包括第一数据端口，第一控制端口和第二控制端口。 所述第一数据端口被配置为接收要由对应于所述第一组件的第一任务处理的数据，或者被配置为提供由对应于所述第一组件的所述第一任务处理的数据。 执行对应于所述基于图形的程序规范的程序包括：响应于接收到所述第一控制信息，确定是否调用所述第一任务，并且在接收到所述第一控制信息之后，在所述第一控制端口处接收所述第一控制信息 从所述第二控制端口提供所述第二控制信息。

公开(公告)号：US20160062749A1

公开(公告)日：2016-03-03

申请号：US14843162

申请日：2015-09-02

Applicant: Ab Initio Technology LLC

Inventor： Craig W. Stanfill ,  Richard Shapiro ,  Adam Weiss ,  Andrew F. Roberts ,  Joseph Skeffington Wholey, III ,  Joel Gould ,  Stephen A. Kukolich

IPC: G06F9/45 ,  G06F13/36

CPC classification number: G06F8/43 ,  G06F8/34 ,  G06F8/433 ,  G06F8/447 ,  G06F9/44505 ,  G06F9/44521 ,  G06F9/4494 ,  G06F9/5066 ,  G06F13/36

Abstract: A graph-based program specification includes components corresponding to tasks and directed links between ports of the components, including: a first type of link configuration between ports of linked components, corresponding to transfer of control or transfer of a single data element, and a second type of link configuration between ports of linked components, corresponding to transfer of multiple data elements. A compiler generates a target program specification including control code representing at least one control graph including graph nodes representing the components, where at least two are connected based on links of the first type. A computing node initiates execution of the target program specification, and manages computing resources for links of the second type, the computing resources including at least one of: (1) a buffer for storing data elements provided by an output port, or (2) a buffer for storing data elements provided to an input port.

Abstract translation: 基于图形的程序规范包括对应于组件的端口之间的任务和定向链接的组件，包括：链接组件的端口之间的第一类型的链路配置，对应于单个数据元素的控制或传输的传送，以及第二 链接组件端口之间的链路配置类型，对应于多个数据元素的传输。 编译器生成包括表示至少一个控制图的控制代码的目标程序规范，所述控制代码包括表示组件的图形节点，其中至少两个基于第一类型的链接被连接。 计算节点启动目标程序规范的执行，并且管理第二类型的链路的计算资源，所述计算资源包括以下各项中的至少一个：（1）用于存储由输出端口提供的数据元素的缓冲器，或者（2） 用于存储提供给输入端口的数据元素的缓冲器。

公开(公告)号：US20160062736A1

公开(公告)日：2016-03-03

申请号：US14843084

申请日：2015-09-02

Applicant: Ab Initio Technology LLC

Inventor： Craig W. Stanfill ,  Adam Weiss ,  Andrew F. Roberts ,  Stephen A. Kukolich

IPC: G06F9/44 ,  G06F3/0482

CPC classification number: G06F8/10 ,  G06F3/0482 ,  G06F8/34 ,  G06F8/433 ,  G06F8/456 ,  G06F9/4494

Abstract: User input is received specifying components of a graph-based program specification. User input is received specifying links, at least some connecting an output port of an upstream component to an input port of a downstream component. The graph-based program specification is processed to identify one or more subsets of the components, including: identifying one or more subset entry points and one or more subset exit points that occur between components in different subsets based at least in part on data processing characteristics of linked components, and forming the subsets based on the identified subset entry points and exit points. A visual representation of the formed subsets is rendered within a user interface. Prepared code is generated for each formed subset that when used for execution by a runtime system causes processing tasks corresponding to the components in each formed subset to be performed.

Abstract translation: 接收到用户输入，指定基于图表的程序规范的组件。 接收用户输入指定链路，至少一些将上游组件的输出端口连接到下游组件的输入端口。 处理基于图形的程序规范以识别组件的一个或多个子集，包括：至少部分地基于数据处理特性来识别出现在不同子集中的组件之间的一个或多个子集入口点和一个或多个子集出口点 并且基于所识别的子集入口点和出口点形成子集。 形成的子集的视觉表示在用户界面内呈现。 为每个形成的子集生成准备代码，当用于运行时系统的执行时，将导致与每个形成的子集中的组件相对应的处理任务。

公开(公告)号：US09251225B2

公开(公告)日：2016-02-02

申请号：US13949761

申请日：2013-07-24

Applicant: Ab Initio Technology LLC

Inventor： Craig W. Stanfill

IPC: G06F17/30

CPC classification number: G06F17/30557

Abstract: Mapping information that specifies attributes of destination entities in terms of attributes of source entities is received. At least some source entities correspond to respective sets of records in one or more data storage systems. The mapping information is processed to generate a procedural specification for computing values corresponding to attributes of destination entities. Collections of nodes are generated, each including a first node representing a first relational expression associated with a specified attribute. At least some collections form a directed acyclic graph that includes links to one or more other nodes representing respective relational expressions associated with at least one attribute of at least one source entity referenced by a relational expression of a node in the graph. At least two of the collections are merged with each other to form a third collection based on comparing relational expressions of nodes being merged.

Abstract translation: 接收到根据源实体的属性指定目的地实体的属性的映射信息。 至少一些源实体对应于一个或多个数据存储系统中的相应记录集。 处理映射信息以生成用于计算与目的地实体的属性对应的值的过程规范。 生成节点的集合，每个节点包括表示与指定属性相关联的第一关系表达式的第一节点。 至少一些集合形成有向非循环图，其包括指向与图中的节点的关系表达式引用的至少一个源实体的至少一个属性相关联的相应关系表达的一个或多个其他节点的链接。 基于比较被合并的节点的关系表达式，至少两个集合彼此合并以形成第三个集合。