- Patent Title: Autoscaling using file access or cache usage for cluster machines
-
Application No.: US17020573Application Date: 2020-09-14
-
Publication No.: US11379272B2Publication Date: 2022-07-05
- Inventor: Srinath Shankar , Eric Keng-Hao Liang
- Applicant: Databricks Inc.
- Applicant Address: US CA San Francisco
- Assignee: Databricks Inc.
- Current Assignee: Databricks Inc.
- Current Assignee Address: US CA San Francisco
- Agency: Van Pelt, Yi & James LLP
- Main IPC: G06F9/46
- IPC: G06F9/46 ; G06F9/50 ; G06F9/48 ; G06F9/38 ; H04L67/5682

Abstract:
The allocation system comprises an interface and a processor. The interface is configured to receive an indication to deactivate idle cluster machines of a set of cluster machines. The processor is configured to determine a list of cluster machines storing one or more intermediate data files of a set of intermediate data files; determine a set of idle cluster machines of the set of cluster machines that are neither running one or more tasks of a set of tasks executing or pending on the set of cluster machines nor storing the one or more intermediate data files of the set of intermediate data files, where the set of intermediate data files is associated with the set of tasks executing or pending on the cluster machines; and deactivate each cluster machine of the set of idle cluster machines.
Information query