公开(公告)号：US20240362215A1

公开(公告)日：2024-10-31

申请号：US18140323

申请日：2023-04-27

Applicant: Databricks, Inc.

Inventor： Venkata Sai Akhil Gudesa ,  Herman Rudolf Petrus Catharina van Hövell tot Westerflier ,  Supun Chathuranga Nakandala

IPC: G06F16/2453 ,  G06F9/48 ,  G06F11/34 ,  G06F16/28

CPC classification number: G06F16/2453 ,  G06F9/4887 ,  G06F11/3419 ,  G06F16/285

Abstract: A cluster computing system maintains a first set of queues for short queries and a set second set for longer queries. The first set is allocated a majority of the cluster's processing resources and processes queries on a first in first out basis. The second set is allocated a minority of the cluster's processing resources which are shared among queries in the second set. Accordingly, the system assigns each query to the first set of queues for a fixed amount of resource time. While a query is processing, the system monitors the query's resource time and reassigns the query to the second set of queues if the query has not completed within the allotted amount of resource time. Thus, short queries receive the necessary resources to complete quickly without getting stuck behind longer queries while ensuring that longer queries continue to make progress.

公开(公告)号：US12210521B2

公开(公告)日：2025-01-28

申请号：US18140323

申请日：2023-04-27

Applicant: Databricks, Inc.

Inventor： Venkata Sai Akhil Gudesa ,  Herman Rudolf Petrus Catharina van Hövell tot Westerflier ,  Supun Chathuranga Nakandala

IPC: G06F16/24 ,  G06F9/48 ,  G06F11/34 ,  G06F16/2453 ,  G06F16/28

Abstract: A cluster computing system maintains a first set of queues for short queries and a set second set for longer queries. The first set is allocated a majority of the cluster's processing resources and processes queries on a first in first out basis. The second set is allocated a minority of the cluster's processing resources which are shared among queries in the second set. Accordingly, the system assigns each query to the first set of queues for a fixed amount of resource time. While a query is processing, the system monitors the query's resource time and reassigns the query to the second set of queues if the query has not completed within the allotted amount of resource time. Thus, short queries receive the necessary resources to complete quickly without getting stuck behind longer queries while ensuring that longer queries continue to make progress.