公开(公告)号：US11567998B2

公开(公告)日：2023-01-31

申请号：US17362450

申请日：2021-06-29

Applicant: Databricks, Inc.

Inventor： Michael Paul Armbrust ,  Andreas Neumann ,  Mukul Murthy ,  Jonathan Mio

IPC: G06F16/901 ,  G06F16/245 ,  G06F16/22

Abstract: A system for dataflow graph processing comprises a communication interface and a processor. The communication interface is configured receive an indication to generate a dataflow graph, wherein the indication includes a set of queries and/or commands. The processor is coupled to the communication interface and configured to: determine dependencies of each query in the set of queries on another query; determine a DAG of nodes based at least in part on the dependencies; determine the dataflow graph by determining in-line expressions for tables of the dataflow graph aggregating calculations associated with a subset of dataflow graph nodes designated as view nodes; and provide the dataflow graph.

公开(公告)号：US20220309103A1

公开(公告)日：2022-09-29

申请号：US17362450

申请日：2021-06-29

Applicant: Databricks Inc.

Inventor： Michael Paul Armbrust ,  Andreas Neumann ,  Mukul Murthy ,  Jonathan Mio

IPC: G06F16/901 ,  G06F16/22 ,  G06F16/245

Abstract: A system for dataflow graph processing comprises a communication interface and a processor. The communication interface is configured receive an indication to generate a dataflow graph, wherein the indication includes a set of queries and/or commands. The processor is coupled to the communication interface and configured to: determine dependencies of each query in the set of queries on another query; determine a DAG of nodes based at least in part on the dependencies; determine the dataflow graph by determining in-line expressions for tables of the dataflow graph aggregating calculations associated with a subset of dataflow graph nodes designated as view nodes; and provide the dataflow graph.

公开(公告)号：US12019682B2

公开(公告)日：2024-06-25

申请号：US18089349

申请日：2022-12-27

Applicant: Databricks, Inc.

Inventor： Michael Paul Armbrust ,  Andreas Neumann ,  Mukul Murthy ,  Jonathan Mio

IPC: G06F16/901 ,  G06F16/215 ,  G06F16/22 ,  G06F16/245

CPC classification number: G06F16/9024 ,  G06F16/215 ,  G06F16/2282 ,  G06F16/245

Abstract: A system for dataflow graph processing comprises a communication interface and a processor. The communication interface is configured receive an indication to generate a dataflow graph, wherein the indication includes a set of queries and/or commands. The processor is coupled to the communication interface and configured to: determine dependencies of each query in the set of queries on another query; determine a DAG of nodes based at least in part on the dependencies; determine the dataflow graph by determining in-line expressions for tables of the dataflow graph aggregating calculations associated with a subset of dataflow graph nodes designated as view nodes; and provide the dataflow graph.

公开(公告)号：US12008040B2

公开(公告)日：2024-06-11

申请号：US17362456

申请日：2021-06-29

Applicant: Databricks, Inc.

Inventor： Michael Paul Armbrust ,  Andreas Neumann ,  Mukul Murthy ,  Jonathan Mio

IPC: G06F16/901 ,  G06F16/215 ,  G06F16/22 ,  G06F16/245

CPC classification number: G06F16/9024 ,  G06F16/215 ,  G06F16/2282 ,  G06F16/245

Abstract: A system for dataflow graph processing comprises a communication interface and a processor. The communication interface is configured receive an indication to generate a dataflow graph, wherein the indication includes a set of queries. The processor is coupled to the communication interface and is configured to: determine dependencies of each query in the set of queries on another query; determine a DAG of nodes based at least in part on the dependencies; insert a node in the DAG of nodes to generate an updated DAG to enforce an expectation; determine a dataflow graph based on the updated DAG; and provide the dataflow graph.

公开(公告)号：US20250005076A1

公开(公告)日：2025-01-02

申请号：US18658418

申请日：2024-05-08

Applicant: Databricks, Inc.

Inventor： Michael Paul Armbrust ,  Andreas Neumann ,  Mukul Murthy ,  Jonathan Mio

IPC: G06F16/901 ,  G06F16/215 ,  G06F16/22 ,  G06F16/245

Abstract: A system for dataflow graph processing comprises a communication interface and a processor. The communication interface is configured receive an indication to generate a dataflow graph, wherein the indication includes a set of queries. The processor is coupled to the communication interface and is configured to: determine dependencies of each query in the set of queries on another query; determine a DAG of nodes based at least in part on the dependencies; insert a node in the DAG of nodes to generate an updated DAG to enforce an expectation; determine a dataflow graph based on the updated DAG; and provide the dataflow graph.

公开(公告)号：US20230140169A1

公开(公告)日：2023-05-04

申请号：US18089349

申请日：2022-12-27

Applicant: Databricks, Inc.

Inventor： Michael Paul Armbrust ,  Andreas Neumann ,  Mukul Murthy ,  Jonathan Mio

IPC: G06F16/901 ,  G06F16/245 ,  G06F16/215 ,  G06F16/22

Abstract: A system for dataflow graph processing comprises a communication interface and a processor. The communication interface is configured receive an indication to generate a dataflow graph, wherein the indication includes a set of queries and/or commands. The processor is coupled to the communication interface and configured to: determine dependencies of each query in the set of queries on another query; determine a DAG of nodes based at least in part on the dependencies; determine the dataflow graph by determining in-line expressions for tables of the dataflow graph aggregating calculations associated with a subset of dataflow graph nodes designated as view nodes; and provide the dataflow graph.

公开(公告)号：US20220309104A1

公开(公告)日：2022-09-29

申请号：US17362456

申请日：2021-06-29

Applicant: Databricks Inc.

Inventor： Michael Paul Armbrust ,  Andreas Neumann ,  Mukul Murthy ,  Jonathan Mio

IPC: G06F16/901 ,  G06F16/215 ,  G06F16/245

Abstract: A system for dataflow graph processing comprises a communication interface and a processor. The communication interface is configured receive an indication to generate a dataflow graph, wherein the indication includes a set of queries. The processor is coupled to the communication interface and is configured to: determine dependencies of each query in the set of queries on another query; determine a DAG of nodes based at least in part on the dependencies; insert a node in the DAG of nodes to generate an updated DAG to enforce an expectation; determine a dataflow graph based on the updated DAG; and provide the dataflow graph.