公开(公告)号：US11709866B2

公开(公告)日：2023-07-25

申请号：US17463353

申请日：2021-08-31

Applicant: Snowflake Inc.

Inventor： Chen Luo ,  Alexander Miller ,  William Waddington ,  Zhaohui Zhang

IPC: G06F16/27 ,  G06F16/23

CPC classification number: G06F16/273 ,  G06F16/2343 ,  G06F16/2379

Abstract: A distributed database compaction system can perform compaction for transactional queries in an asynchronous matter without affecting completion of the queries. The compaction system can implement asynchronous transformation of key pairs in the database, and older keys can be periodically purged using a scheduled compactor. Subsequent queries use the compacted stored keys to perform efficient queries with direct reads of committed transactions and more efficient access to key values stores of the distributed database.

公开(公告)号：US20230325409A1

公开(公告)日：2023-10-12

申请号：US18326683

申请日：2023-05-31

Applicant: Snowflake Inc.

Inventor： Chen Luo ,  Alexander Miller ,  William Waddington ,  Zhaohui Zhang

IPC: G06F16/27 ,  G06F16/23

CPC classification number: G06F16/273 ,  G06F16/2343 ,  G06F16/2379

Abstract: The subject technology performs, using asynchronous threads of execution nodes of a distributed database, asynchronous transformation and compaction of key-value pairs of data that correspond to a plurality of transactional queries, the asynchronous threads transforming the key-value pairs from an initial format to a commit format that includes a commit time of data committed to the distributed database, the asynchronous threads compacting the key-value pairs by deleting the key-value pairs in the initial format. The subject technology processes, using transactional threads in the execution nodes, the plurality of transactional queries to generate results data, the asynchronous threads performing the asynchronous transformation and compaction while the transactional threads generate the results data. The subject technology stores the results data.

公开(公告)号：US20230195719A1

公开(公告)日：2023-06-22

申请号：US17555197

申请日：2021-12-17

Applicant: Snowflake Inc.

Inventor： Chen Luo ,  Alexander Miller ,  William Waddington ,  Mingze Xu

IPC: G06F16/23 ,  G06F16/27

CPC classification number: G06F16/2379 ,  G06F16/27

Abstract: The subject technology receives a key-value pair. The subject technology determines a value size of the key-value pair is greater than a threshold size for an out of line blob (OLB) key space. The subject technology performs a write operation of a key-pointer pair into a regular space. The subject technology performs a plurality of write operations for multiple key-value parts of the key-value pair into the OLB key space. The subject technology performs conflict checking of the regular space for the key-value pair.

公开(公告)号：US20230055715A1

公开(公告)日：2023-02-23

申请号：US17463353

申请日：2021-08-31

Applicant: Snowflake Inc.

Inventor： Chen Luo ,  Alexander Miller ,  William Waddington ,  Zhaohui Zhang

IPC: G06F16/27 ,  G06F16/23

Abstract: A distributed database compaction system can perform compaction for transactional queries in an asynchronous matter without affecting completion of the queries. The compaction system can implement asynchronous transformation of key pairs in the database, and older keys can be periodically purged using a scheduled compactor. Subsequent queries use the compacted stored keys to perform efficient queries with direct reads of committed transactions and more efficient access to key values stores of the distributed database.

公开(公告)号：US20220092050A1

公开(公告)日：2022-03-24

申请号：US17454368

申请日：2021-11-10

Applicant: Snowflake Inc.

Inventor： Chen Luo ,  Alexander Miller ,  William Waddington ,  Zhaohui Zhang

IPC: G06F16/23

Abstract: The subject technology inserts, by a first transaction, a new version of an object, the inserting corresponding to a first statement from the first transaction, the first statement including information indicating a first key, a first transaction identifier, a first statement number, and a first restart count. The subject technology performs, by a second transaction, a range read, the range read including information indicating the first key. The subject technology receives, after performing the range read, a set of conflicting transactions. The subject technology determines that a conflict occurred between the first transaction and a third transaction from the set of conflicting transactions.

公开(公告)号：US11243820B1

公开(公告)日：2022-02-08

申请号：US17334946

申请日：2021-05-31

Applicant: Snowflake Inc.

Inventor： Chen Luo ,  Alexander Miller ,  William Waddington

IPC: G06F9/52 ,  G06F16/23 ,  G06F16/25

Abstract: The subject technology performs a locking operation on a first set of keys by a first statement of a first transaction. The subject technology determines that a conflict occurred between the first statement and a second transaction. The subject technology determines that the second transaction has yet to complete after a predetermined period of time. The subject technology performs a deadlock detection process where the subject technology stores a key and value in a table indicating the first transaction and the second transaction, detects, based at least in part on a graph traversal of the table starting from the first transaction, a cycle between the first transaction and the second transaction, and determines that the first transaction is a youngest transaction in the detected cycle. The subject technology ceases execution of the first transaction in response to the first transaction being a youngest transaction in a detected cycle.

公开(公告)号：US12222960B2

公开(公告)日：2025-02-11

申请号：US18326683

申请日：2023-05-31

Applicant: Snowflake Inc.

Inventor： Chen Luo ,  Alexander Miller ,  William Waddington ,  Zhaohui Zhang

IPC: G06F16/27 ,  G06F16/23

Abstract: The subject technology performs, using asynchronous threads of execution nodes of a distributed database, asynchronous transformation and compaction of key-value pairs of data that correspond to a plurality of transactional queries, the asynchronous threads transforming the key-value pairs from an initial format to a commit format that includes a commit time of data committed to the distributed database, the asynchronous threads compacting the key-value pairs by deleting the key-value pairs in the initial format. The subject technology processes, using transactional threads in the execution nodes, the plurality of transactional queries to generate results data, the asynchronous threads performing the asynchronous transformation and compaction while the transactional threads generate the results data. The subject technology stores the results data.

公开(公告)号：US20250005022A1

公开(公告)日：2025-01-02

申请号：US18656008

申请日：2024-05-06

Applicant: Snowflake Inc.

Inventor： Fabio Angius ,  Chen Luo ,  Sarvesh Singh

IPC: G06F16/2453 ,  G06F16/23 ,  G06F16/28

Abstract: Techniques for multi database query processing are described. Objects located in a plurality of databases referenced in a query can be compiled. A connection string based on the compiled objects can be generated. The connection string can include mapping information related to the related to the plurality of databases and cluster information of where the plurality of databases are stored in the network-based data system. The connection string can then be included in a query plan to allow for execution of the query plan using the connection string to access the objects in the plurality of databases.

公开(公告)号：US12086154B1

公开(公告)日：2024-09-10

申请号：US18455229

申请日：2023-08-24

Applicant: Snowflake Inc.

Inventor： Mihir Dharamshi ,  Cristian Diaconu ,  Chen Luo ,  Andrew McCormick ,  Corbin McElhanney ,  Joshua Slocum ,  Wumengjian Zhu

IPC: G06F16/25 ,  G06F16/11 ,  G06F16/172 ,  G06F16/23

CPC classification number: G06F16/254 ,  G06F16/116 ,  G06F16/172 ,  G06F16/2379

Abstract: The subject technology receives a query, the query including a query range for processing the query. The subject technology sends a request to a key-value store for blob metadata and a set of recent writes for the query range. The subject technology receives the blob metadata, the blob metadata including information related to a set of blob files. The subject technology determines whether the set of blob files is stored in a local cache. The subject technology, in response to at least one blob file being missing from the set of blob files, sends a request to a blob store to retrieve the at least one blob file of the set of blob files. The subject technology transforms the retrieved at least one blob file to a column file format. The subject technology stores the transformed at least one blob file in the local cache.

公开(公告)号：US11899648B2

公开(公告)日：2024-02-13

申请号：US17821581

申请日：2022-08-23

Applicant: Snowflake Inc.

Inventor： Chen Luo ,  Alexander Miller ,  William Waddington ,  Zhaohui Zhang

IPC: G06F16/23

CPC classification number: G06F16/2322 ,  G06F16/2365 ,  G06F16/2379

Abstract: The subject technology inserts, by a first transaction, a new version of an object, the first transaction including a first statement to perform an update operation to a row in a first table, the object corresponding to data in the row to be updated, the first statement including information comprising an object key associated with the object. The subject technology performs, by a second transaction, a range read, the range read including information indicating the object key. The subject technology receives a set of conflicting transactions from the range read. The subject technology determines that a conflict occurred between the first transaction and a third transaction from the set of conflicting transactions. The subject technology performs a restart of the first transaction in response to determining that the conflict occurred.