公开(公告)号：US20250013606A1

公开(公告)日：2025-01-09

申请号：US18218410

申请日：2023-07-05

Applicant: Databricks, Inc.

Inventor： Prakhar Jain ,  Frederick Ryan Johnson ,  Terry Kim ,  Vijayan Prabhakaran ,  Bart Samwel

IPC: G06F16/16 ,  G06F16/13

Abstract: A data processing service generates a data classifier tree for managing data files of a data table. The data classifier tree may be configured as a KD-classifier tree and includes a plurality of nodes and edges. A node of the data classifier tree may represent a splitting condition with respect to key-values for a respective key. A node of the data classifier tree may be associated with one or more data files assigned to the node. The data files assigned to the node each include a subset of records having key-values that satisfy the conditions represented by the node and parent nodes of the node. The data processing service may efficiently cluster the data in the data table while reducing the number of data files that are rewritten when data is modified or added to the data table.

公开(公告)号：US20240378181A1

公开(公告)日：2024-11-14

申请号：US18144647

申请日：2023-05-08

Applicant: Databricks, Inc.

Inventor： Vijayan Prabhakaran ,  Himanshu Raja ,  Rahul Potharaju ,  Naga Raju Bhanoori ,  Lin Ma ,  Rajesh Parangi Sharabhalingappa ,  Jintian Liang ,  Zach Schuermann ,  Kam Cheung Ting

IPC: G06F16/21 ,  G06F11/34 ,  G06F16/22

Abstract: Disclosed is a configuration for managing the organization of data tables in cloud-based storage. The configuration receives metrics for data processing operations on the data table. Metrics include at least one of a size of the data table, a size of each file in the data table, and metadata describing the data table. The configuration automatically executes a cost-benefit analysis based on the one or more metrics for each candidate maintenance operation in a plurality of candidate maintenance operations. The configuration automatically selects a maintenance operation from the candidate maintenance operations to automate based on the cost-benefit analysis of the one or more candidate maintenance operations. The selected maintenance operation is automated and scheduled on the data table.

公开(公告)号：US12204510B2

公开(公告)日：2025-01-21

申请号：US18144647

申请日：2023-05-08

Applicant: Databricks, Inc.

Inventor： Vijayan Prabhakaran ,  Himanshu Raja ,  Rahul Potharaju ,  Naga Raju Bhanoori ,  Lin Ma ,  Rajesh Parangi Sharabhalingappa ,  Jintian Liang ,  Zachary Vaughn Schuermann ,  Kam Cheung Ting

IPC: G06F16/21 ,  G06F11/34 ,  G06F16/22

Abstract: Disclosed is a configuration for managing the organization of data tables in cloud-based storage. The configuration receives metrics for data processing operations on the data table. Metrics include at least one of a size of the data table, a size of each file in the data table, and metadata describing the data table. The configuration automatically executes a cost-benefit analysis based on the one or more metrics for each candidate maintenance operation in a plurality of candidate maintenance operations. The configuration automatically selects a maintenance operation from the candidate maintenance operations to automate based on the cost-benefit analysis of the one or more candidate maintenance operations. The selected maintenance operation is automated and scheduled on the data table.