Distributed deadlock detection and resolution in distributed databases

    公开(公告)号:US11243820B1

    公开(公告)日:2022-02-08

    申请号:US17334946

    申请日:2021-05-31

    Applicant: Snowflake Inc.

    Abstract: The subject technology performs a locking operation on a first set of keys by a first statement of a first transaction. The subject technology determines that a conflict occurred between the first statement and a second transaction. The subject technology determines that the second transaction has yet to complete after a predetermined period of time. The subject technology performs a deadlock detection process where the subject technology stores a key and value in a table indicating the first transaction and the second transaction, detects, based at least in part on a graph traversal of the table starting from the first transaction, a cycle between the first transaction and the second transaction, and determines that the first transaction is a youngest transaction in the detected cycle. The subject technology ceases execution of the first transaction in response to the first transaction being a youngest transaction in a detected cycle.

    Internal resource provisioning in database systems

    公开(公告)号:US11138213B2

    公开(公告)日:2021-10-05

    申请号:US16380848

    申请日:2019-04-10

    Applicant: Snowflake Inc.

    Abstract: Resource provisioning systems and methods are described. In an embodiment, a system includes a plurality of shared storage devices collectively storing database data, an execution platform, and a compute service manager. The compute service manager is configured to determine a task to be executed in response to a trigger event and determine a query plan for executing the task, wherein the query plan comprises a plurality of discrete subtasks. The compute service manager is further configured to assign the plurality of discrete subtasks to one or more nodes of a plurality of nodes of the execution platform, determine whether execution of the task is complete, and in response to determining the execution of the task is complete, store a record in the plurality of shared storage devices indicating the task was completed.

    Transferring metadata from immutable storage

    公开(公告)号:US11048687B2

    公开(公告)日:2021-06-29

    申请号:US17086245

    申请日:2020-10-30

    Applicant: Snowflake Inc.

    Abstract: The subject technology generates and stores a new version set of one or more table-metadata files, the new version set of one or more table-metadata files comprising table metadata for a new version of a database table. The subject technology determines that a plurality of table-metadata files are not included in a cache. The subject technology downloads, in parallel, the plurality of table-metadata files from immutable storage. The subject technology stores, in the cache, the plurality of table-metadata files. The subject technology reads, among the plurality of table-metadata files, a first table-metadata file before a second table-metadata file has been fully downloaded, the plurality of table-metadata files comprising at least the first table-metadata file and the second table-metadata file.

    Providing new table metadata
    24.
    发明授权

    公开(公告)号:US10997157B2

    公开(公告)日:2021-05-04

    申请号:US16944153

    申请日:2020-07-31

    Applicant: Snowflake Inc.

    Abstract: The subject technology stores table metadata for a current version of a table in a current-version set of one or more table-metadata files. The subject technology detects a modification to the current version of the table, the modification resulting in a new version of the table, the new version of the table being stored in a new-version set of one or more table-data files, the new-version set of one or more table-data files being different than a current-version set of one or more table-data files. The subject technology, responsive to detecting the modification to the current version of the table, generates and storing a new-version set of one or more table-metadata files, the new-version set of one or more table-metadata files comprising table metadata for the new version of the table

    Reclustering of database tables based on peaks and widths

    公开(公告)号:US10956394B2

    公开(公告)日:2021-03-23

    申请号:US16941215

    申请日:2020-07-28

    Applicant: Snowflake Inc.

    Abstract: The subject technology determines whether a table is sufficiently clustered. The subject technology in response to determining the table is not sufficiently clustered, selects one or more micro-partitions of the table to be reclustered. The subject technology constructs a data structure for the table. The subject technology extracts minimum and maximum endpoints for each micro-partition in the data structure. The subject technology sorts each of one or more peaks in the data structure based on height. The subject technology sorts overlapping micro-partitions based on width. The subject technology selects based on which micro-partitions are within the tallest peaks of the one or more peaks and further based on which of the overlapping micro-partitions have the widest widths.

    MANAGING VERSIONS OF DATABASE METADATA IN STORAGE

    公开(公告)号:US20200327115A1

    公开(公告)日:2020-10-15

    申请号:US16915435

    申请日:2020-06-29

    Applicant: Snowflake Inc.

    Abstract: A method for a database system includes storing table data for a database, the table data including information in rows and columns of one or more database tables. The method includes storing metadata on immutable storage, the metadata including information about the table data for the database. In one embodiment, mutable metadata may be periodically consolidated in the background to create new versions of metadata files and which allows for deletions of old metadata files and old data files.

    ASYNCHRONOUS TASK QUEUE CONFIGURATION IN A DATABASE SYSTEM

    公开(公告)号:US20250036464A1

    公开(公告)日:2025-01-30

    申请号:US18670459

    申请日:2024-05-21

    Applicant: Snowflake Inc.

    Abstract: Techniques for configuring an asynchronous task queue are disclosed. An example method includes enqueuing, by at least one hardware processor of a first computing node, a task in a task queue. The at least one hardware processor detects a lease request for the task. The lease request is received from a second computing node. The at least one hardware processor initiates a lease of the task to the second computing node. The at least one hardware processor updates a state of the task based on the lease. The at least one hardware processor determines whether to dequeue the task based on a task processing update received from the second computing node.

    Serialization of data in a concurrent transaction processing distributed database

    公开(公告)号:US12189614B2

    公开(公告)日:2025-01-07

    申请号:US18477834

    申请日:2023-09-29

    Applicant: Snowflake Inc.

    Abstract: The subject technology serializes, by at least one hardware processor, non-primary key data of column-organized data into compressed serialized value data that is in a row-organized sequence, the compressed serialized value data compressed using at least one bitmap, the non-primary key data comprising a schema identifier, the column-organized data being stored in a columnar database system, the column-organized data comprising primary key data and the non-primary key data. The subject technology stores the compressed serialized value data in a key-value data store of a key-value database system, the key-value database system processing key-value data in a key-value format. The subject technology receives a query by the columnar database system. The subject technology deserializes a portion of the compressed serialized value data that corresponds to the query. The subject technology processes the query using the columnar database system.

Patent Agency Ranking