Invention Publication
- Patent Title: SHORT QUERY PRIORITIZATION FOR DATA PROCESSING SERVICE
-
Application No.: US18140323Application Date: 2023-04-27
-
Publication No.: US20240362215A1Publication Date: 2024-10-31
- Inventor: Venkata Sai Akhil Gudesa , Herman Rudolf Petrus Catharina van Hövell tot Westerflier , Supun Chathuranga Nakandala
- Applicant: Databricks, Inc.
- Applicant Address: US CA San Francisco
- Assignee: Databricks, Inc.
- Current Assignee: Databricks, Inc.
- Current Assignee Address: US CA San Francisco
- Main IPC: G06F16/2453
- IPC: G06F16/2453 ; G06F9/48 ; G06F11/34 ; G06F16/28

Abstract:
A cluster computing system maintains a first set of queues for short queries and a set second set for longer queries. The first set is allocated a majority of the cluster's processing resources and processes queries on a first in first out basis. The second set is allocated a minority of the cluster's processing resources which are shared among queries in the second set. Accordingly, the system assigns each query to the first set of queues for a fixed amount of resource time. While a query is processing, the system monitors the query's resource time and reassigns the query to the second set of queues if the query has not completed within the allotted amount of resource time. Thus, short queries receive the necessary resources to complete quickly without getting stuck behind longer queries while ensuring that longer queries continue to make progress.
Public/Granted literature
- US12210521B2 Short query prioritization for data processing service Public/Granted day:2025-01-28
Information query