-
公开(公告)号:US11243820B1
公开(公告)日:2022-02-08
申请号:US17334946
申请日:2021-05-31
Applicant: Snowflake Inc.
Inventor: Chen Luo , Alexander Miller , William Waddington
Abstract: The subject technology performs a locking operation on a first set of keys by a first statement of a first transaction. The subject technology determines that a conflict occurred between the first statement and a second transaction. The subject technology determines that the second transaction has yet to complete after a predetermined period of time. The subject technology performs a deadlock detection process where the subject technology stores a key and value in a table indicating the first transaction and the second transaction, detects, based at least in part on a graph traversal of the table starting from the first transaction, a cycle between the first transaction and the second transaction, and determines that the first transaction is a youngest transaction in the detected cycle. The subject technology ceases execution of the first transaction in response to the first transaction being a youngest transaction in a detected cycle.
-
公开(公告)号:US11138213B2
公开(公告)日:2021-10-05
申请号:US16380848
申请日:2019-04-10
Applicant: Snowflake Inc.
Inventor: Jeffrey Rosen , Abdul Munir , Jiaqi Yan , William Waddington , Prasanna Rajaperumal , Thierry Cruanes
IPC: G06F16/24 , G06F16/2458 , G06F16/2453 , G06F9/50 , G06F16/2455
Abstract: Resource provisioning systems and methods are described. In an embodiment, a system includes a plurality of shared storage devices collectively storing database data, an execution platform, and a compute service manager. The compute service manager is configured to determine a task to be executed in response to a trigger event and determine a query plan for executing the task, wherein the query plan comprises a plurality of discrete subtasks. The compute service manager is further configured to assign the plurality of discrete subtasks to one or more nodes of a plurality of nodes of the execution platform, determine whether execution of the task is complete, and in response to determining the execution of the task is complete, store a record in the plurality of shared storage devices indicating the task was completed.
-
公开(公告)号:US11048687B2
公开(公告)日:2021-06-29
申请号:US17086245
申请日:2020-10-30
Applicant: Snowflake Inc.
Inventor: Benoit Dageville , Martin Hentschel , William Waddington
IPC: G06F16/00 , G06F16/23 , G06F21/60 , G06F16/22 , G06F16/2455
Abstract: The subject technology generates and stores a new version set of one or more table-metadata files, the new version set of one or more table-metadata files comprising table metadata for a new version of a database table. The subject technology determines that a plurality of table-metadata files are not included in a cache. The subject technology downloads, in parallel, the plurality of table-metadata files from immutable storage. The subject technology stores, in the cache, the plurality of table-metadata files. The subject technology reads, among the plurality of table-metadata files, a first table-metadata file before a second table-metadata file has been fully downloaded, the plurality of table-metadata files comprising at least the first table-metadata file and the second table-metadata file.
-
公开(公告)号:US10997157B2
公开(公告)日:2021-05-04
申请号:US16944153
申请日:2020-07-31
Applicant: Snowflake Inc.
Inventor: Benoit Dageville , Martin Hentschel , William Waddington
IPC: G06F16/00 , G06F16/23 , G06F21/60 , G06F16/22 , G06F16/2455
Abstract: The subject technology stores table metadata for a current version of a table in a current-version set of one or more table-metadata files. The subject technology detects a modification to the current version of the table, the modification resulting in a new version of the table, the new version of the table being stored in a new-version set of one or more table-data files, the new-version set of one or more table-data files being different than a current-version set of one or more table-data files. The subject technology, responsive to detecting the modification to the current version of the table, generates and storing a new-version set of one or more table-metadata files, the new-version set of one or more table-metadata files comprising table metadata for the new version of the table
-
公开(公告)号:US10963443B2
公开(公告)日:2021-03-30
申请号:US16918591
申请日:2020-07-01
Applicant: Snowflake Inc.
Inventor: Jiaqi Yan , Thierry Cruanes , Jeffrey Rosen , William Waddington , Prasanna Rajaperumal , Abdul Munir
Abstract: Automatic clustering of a database table is disclosed. A method for automatic clustering of a database table includes receiving an indication that a data modification task has been executed on a table and determining whether the table is sufficiently clustered. The method includes, in response to determining the table is not sufficiently clustered, selecting one or more micro-partitions of the table to be reclustered. The method includes assigning each of the one or more micro-partitions to an execution node to be reclustered.
-
公开(公告)号:US10956394B2
公开(公告)日:2021-03-23
申请号:US16941215
申请日:2020-07-28
Applicant: Snowflake Inc.
Inventor: Jiaqi Yan , Thierry Cruanes , Jeffrey Rosen , William Waddington , Prasanna Rajaperumal , Abdul Munir
Abstract: The subject technology determines whether a table is sufficiently clustered. The subject technology in response to determining the table is not sufficiently clustered, selects one or more micro-partitions of the table to be reclustered. The subject technology constructs a data structure for the table. The subject technology extracts minimum and maximum endpoints for each micro-partition in the data structure. The subject technology sorts each of one or more peaks in the data structure based on height. The subject technology sorts overlapping micro-partitions based on width. The subject technology selects based on which micro-partitions are within the tallest peaks of the one or more peaks and further based on which of the overlapping micro-partitions have the widest widths.
-
公开(公告)号:US20200327115A1
公开(公告)日:2020-10-15
申请号:US16915435
申请日:2020-06-29
Applicant: Snowflake Inc.
Inventor: Benoit Dageville , Martin Hentschel , William Waddington
IPC: G06F16/23 , G06F21/60 , G06F16/22 , G06F16/2455
Abstract: A method for a database system includes storing table data for a database, the table data including information in rows and columns of one or more database tables. The method includes storing metadata on immutable storage, the metadata including information about the table data for the database. In one embodiment, mutable metadata may be periodically consolidated in the background to create new versions of metadata files and which allows for deletions of old metadata files and old data files.
-
公开(公告)号:US20250036464A1
公开(公告)日:2025-01-30
申请号:US18670459
申请日:2024-05-21
Applicant: Snowflake Inc.
Inventor: Gabriel Kliot , Ruji Xie , Subramanian Muralidhar , William Waddington
IPC: G06F9/48
Abstract: Techniques for configuring an asynchronous task queue are disclosed. An example method includes enqueuing, by at least one hardware processor of a first computing node, a task in a task queue. The at least one hardware processor detects a lease request for the task. The lease request is received from a second computing node. The at least one hardware processor initiates a lease of the task to the second computing node. The at least one hardware processor updates a state of the task based on the lease. The at least one hardware processor determines whether to dequeue the task based on a task processing update received from the second computing node.
-
公开(公告)号:US12189614B2
公开(公告)日:2025-01-07
申请号:US18477834
申请日:2023-09-29
Applicant: Snowflake Inc.
Inventor: Khaled Yagoub , Wumengjian Zhu , Benoit Dageville , William Waddington
Abstract: The subject technology serializes, by at least one hardware processor, non-primary key data of column-organized data into compressed serialized value data that is in a row-organized sequence, the compressed serialized value data compressed using at least one bitmap, the non-primary key data comprising a schema identifier, the column-organized data being stored in a columnar database system, the column-organized data comprising primary key data and the non-primary key data. The subject technology stores the compressed serialized value data in a key-value data store of a key-value database system, the key-value database system processing key-value data in a key-value format. The subject technology receives a query by the columnar database system. The subject technology deserializes a portion of the compressed serialized value data that corresponds to the query. The subject technology processes the query using the columnar database system.
-
公开(公告)号:US20240345877A1
公开(公告)日:2024-10-17
申请号:US18300738
申请日:2023-04-14
Applicant: Snowflake Inc.
Inventor: Igor Demura , Brian Charles Forney , Weili Gu , Mohammad Saber Mirzaei , William Waddington , Jiaqi Yan
CPC classification number: G06F9/4881 , G06F9/547
Abstract: Techniques for creating child jobs in network-based data systems are described. A request to create a child job for a parent job can be received at a first compute service manager. A child job token can be generated, and a request message can be generated including the child job token and configuration information. The request message can be transmitted to a second compute service manager, where the child job can be constructed linking the child job to the parent job based on the child job token and configuration information. A response message can be received from the second compute service manager including confirmation that the child job was created.
-
-
-
-
-
-
-
-
-