公开(公告)号：US11514045B2

公开(公告)日：2022-11-29

申请号：US16721402

申请日：2019-12-19

Applicant: Databricks Inc.

Inventor： Michael Paul Armbrust ,  Tathagata Das ,  Shi Xin ,  Matei Zaharia

IPC: G06F16/2453 ,  G06F16/2455

Abstract: A system for executing a streaming query includes an interface and a processor. The interface is configured to receive a logical query plan. The processor is configured to determine a physical query plan based at least in part on the logical query plan. The physical query plan comprises an ordered set of operators. Each operator of the ordered set of operators comprises an operator input mode and an operator output mode. The processor is further configured to execute the physical query plan using the operator input mode and the operator output mode for each operator of the query.

公开(公告)号：US11216324B2

公开(公告)日：2022-01-04

申请号：US16793921

申请日：2020-02-18

Applicant: Databricks Inc.

Inventor： Alicja Luszczak ,  Srinath Shankar ,  Shi Xin

IPC: G06F11/07 ,  G06F11/34 ,  G06F11/30

Abstract: A system for monitoring job execution includes an interface and a processor. The interface is configured to receive an indication to start a cluster processing job. The processor is configured to determine whether processing a data instance associated with the cluster processing job satisfies a watchdog criterion; and in the event that processing the data instance satisfies the watchdog criterion, cause the processing of the data instance to be killed.

公开(公告)号：US20200257689A1

公开(公告)日：2020-08-13

申请号：US16721402

申请日：2019-12-19

Applicant: Databricks Inc.

Inventor： Michael Paul Armbrust ,  Tathagata Das ,  Shi Xin ,  Matei Zaharia

IPC: G06F16/2453 ,  G06F16/2455

Abstract: A system for executing a streaming query includes an interface and a processor. The interface is configured to receive a logical query plan. The processor is configured to determine a physical query plan based at least in part on the logical query plan. The physical query plan comprises an ordered set of operators. Each operator of the ordered set of operators comprises an operator input mode and an operator output mode. The processor is further configured to execute the physical query plan using the operator input mode and the operator output mode for each operator of the query.

公开(公告)号：US10606675B1

公开(公告)日：2020-03-31

申请号：US15809907

申请日：2017-11-10

Applicant: Databricks Inc.

Inventor： Alicja Luszczak ,  Srinath Shankar ,  Shi Xin

IPC: G06F11/07 ,  G06F11/34 ,  G06F11/30

Abstract: A system for monitoring job execution includes an interface and a processor. The interface is configured to receive an indication to start a cluster processing job. The processor is configured to determine whether processing a data instance associated with the cluster processing job satisfies a watchdog criterion; and in the event that processing the data instance satisfies the watchdog criterion, cause the processing of the data instance to be killed.

公开(公告)号：US11948084B1

公开(公告)日：2024-04-02

申请号：US18162291

申请日：2023-01-31

Applicant: Databricks, Inc.

Inventor： Sue Ann Hong ,  Shi Xin ,  Timothee Hunter ,  Ali Ghodsi

IPC: G06N3/08 ,  G06N3/04 ,  G06N3/063 ,  G06N5/02 ,  G06N5/022 ,  G06F16/14 ,  G06F16/22

CPC classification number: G06N3/08 ,  G06N3/04 ,  G06N3/063 ,  G06N5/022 ,  G06N5/027 ,  G06F16/14 ,  G06F16/22

Abstract: A function creation method is disclosed. The method comprises defining one or more database function inputs, defining cluster processing information, defining a deep learning model, and defining one or more database function outputs. A database function is created based at least in part on the one or more database function inputs, the cluster set-up information, the deep learning model, and the one or more database function outputs. In some embodiments, the database function enables a non-technical user to utilize deep learning models.

公开(公告)号：US11068447B2

公开(公告)日：2021-07-20

申请号：US15487896

申请日：2017-04-14

Applicant: Databricks Inc.

Inventor： Eric Keng-hao Liang ,  Srinath Shankar ,  Shi Xin

IPC: G06F16/18 ,  G06F16/16 ,  G06F16/182

Abstract: A system for directory level atomic commits includes an interface and a processor. The interface is configured to receive an indication to provide a set of files. The processor is configured to determine whether a file in a directory has been either 1) atomically committed or 2) written by a non-atomic process and not designated as deleted and provide the file as one file of the set of files in the event that the file in the directory has been either 1) atomically committed or 2) written by a non-atomic process and not designated as deleted.

公开(公告)号：US20200241950A1

公开(公告)日：2020-07-30

申请号：US16793921

申请日：2020-02-18

Applicant: Databricks Inc.

Inventor： Alicja Luszczak ,  Srinath Shankar ,  Shi Xin

IPC: G06F11/07 ,  G06F11/34 ,  G06F11/30

Abstract: A system for monitoring job execution includes an interface and a processor. The interface is configured to receive an indication to start a cluster processing job. The processor is configured to determine whether processing a data instance associated with the cluster processing job satisfies a watchdog criterion; and in the event that processing the data instance satisfies the watchdog criterion, cause the processing of the data instance to be killed.

公开(公告)号：US20240394271A1

公开(公告)日：2024-11-28

申请号：US18614380

申请日：2024-03-22

Applicant: Databricks, Inc.

Inventor： Bogdan Ionut Ghit ,  Juliusz Sompolski ,  Shi Xin ,  Bart Samwel

IPC: G06F16/2458 ,  G06F11/34 ,  G06F16/242 ,  G06F16/25

Abstract: The system is configured to: 1) receive a client request; 2) determine executor(s) to generate a response to the user request; 3) provide each of the executor(s) with an indication; 4) receive for each indication a response including an output of either a cloud output or an in-line output to generate a group of in-line outputs and a group of cloud outputs; 5) determine whether the group of in-line outputs comprises all outputs; and 6) in response to the group of in-line outputs not comprising all the outputs for the client request: a) convert the group of in-line outputs to a converted group of cloud outputs; b) generate metadata for the converted group of cloud outputs and the group of cloud outputs; and c) provide response to the client request including the metadata for the converted group of cloud outputs and the group of cloud outputs.

公开(公告)号：US12032573B2

公开(公告)日：2024-07-09

申请号：US17976361

申请日：2022-10-28

Applicant: Databricks, Inc.

Inventor： Michael Paul Armbrust ,  Tathagata Das ,  Shi Xin ,  Matei Zaharia

IPC: G06F16/2453 ,  G06F16/2455

CPC classification number: G06F16/24542 ,  G06F16/24568

Abstract: A system for executing a streaming query includes an interface and a processor. The interface is configured to receive a logical query plan. The processor is configured to determine a physical query plan based at least in part on the logical query plan. The physical query plan comprises an ordered set of operators. Each operator of the ordered set of operators comprises an operator input mode and an operator output mode. The processor is further configured to execute the physical query plan using the operator input mode and the operator output mode for each operator of the query.

公开(公告)号：US11960494B1

公开(公告)日：2024-04-16

申请号：US17841946

申请日：2022-06-16

Applicant: Databricks, Inc.

Inventor： Bogdan Ionut Ghit ,  Juliusz Sompolski ,  Shi Xin ,  Bart Samwel

IPC: G06F16/2458 ,  G06F11/34 ,  G06F16/242 ,  G06F16/25

CPC classification number: G06F16/2471 ,  G06F11/3419 ,  G06F16/244 ,  G06F16/256

Abstract: The system is configured to: 1) receive a client request; 2) determine executor(s) to generate a response to the user request; 3) provide each of the executor(s) with an indication; 4) receive for each indication a response including an output of either a cloud output or an in-line output to generate a group of in-line outputs and a group of cloud outputs; 5) determine whether the group of in-line outputs comprises all outputs; and 6) in response to the group of in-line outputs not comprising all the outputs for the client request: a) convert the group of in-line outputs to a converted group of cloud outputs; b) generate metadata for the converted group of cloud outputs and the group of cloud outputs; and c) provide response to the client request including the metadata for the converted group of cloud outputs and the group of cloud outputs.