公开(公告)号：US20250013606A1

公开(公告)日：2025-01-09

申请号：US18218410

申请日：2023-07-05

Applicant: Databricks, Inc.

Inventor： Prakhar Jain ,  Frederick Ryan Johnson ,  Terry Kim ,  Vijayan Prabhakaran ,  Bart Samwel

IPC: G06F16/16 ,  G06F16/13

Abstract: A data processing service generates a data classifier tree for managing data files of a data table. The data classifier tree may be configured as a KD-classifier tree and includes a plurality of nodes and edges. A node of the data classifier tree may represent a splitting condition with respect to key-values for a respective key. A node of the data classifier tree may be associated with one or more data files assigned to the node. The data files assigned to the node each include a subset of records having key-values that satisfy the conditions represented by the node and parent nodes of the node. The data processing service may efficiently cluster the data in the data table while reducing the number of data files that are rewritten when data is modified or added to the data table.

公开(公告)号：US12229169B1

公开(公告)日：2025-02-18

申请号：US18501830

申请日：2023-11-03

Applicant: Databricks, Inc.

Inventor： Terry Kim ,  Lin Ma ,  Rahul Shivu Mahadev ,  Rahul Potharaju

IPC: G06F16/28 ,  G06F16/21 ,  G06F16/22

Abstract: The disclosed configurations provide a method (and/or a computer-readable medium or system) for determining, from a table schema describing keys of a data table, one or more clustering keys that can be used to cluster data files of a data table. The method includes generating features for the data table, generating tokens from the features, generating a prediction for each token by applying to the token a machine-learned transformer model trained to predict a likelihood that the key associated with the token is a clustering key for the data table, determining clustering keys based on the predictions, and clustering data records of the data table into data files based on key-values for the clustering keys.