公开(公告)号：US20210073206A1

公开(公告)日：2021-03-11

申请号：US17086245

申请日：2020-10-30

Applicant: Snowflake Inc.

Inventor： Benoit Dageville ,  Martin Hentschel ,  William Waddington

IPC: G06F16/23 ,  G06F21/60 ,  G06F16/22 ,  G06F16/2455

Abstract: The subject technology generates and stores a new version set of one or more table-metadata files, the new version set of one or more table-metadata files comprising table metadata for a new version of a database table. The subject technology determines that a plurality of table-metadata files are not included in a cache. The subject technology downloads, in parallel, the plurality of table-metadata files from immutable storage. The subject technology stores, in the cache, the plurality of table-metadata files. The subject technology reads, among the plurality of table-metadata files, a first table-metadata file before a second table-metadata file has been fully downloaded, the plurality of table-metadata files comprising at least the first table-metadata file and the second table-metadata file.

公开(公告)号：US20240394244A1

公开(公告)日：2024-11-28

申请号：US18790664

申请日：2024-07-31

Applicant: Snowflake Inc.

Inventor： Cristian Diaconu ,  Nikolaos Romanos Katsipoulakis ,  Alexander Miller ,  Dimitrios Tsirogiannis ,  William Waddington ,  Zhaohui Zhang

IPC: G06F16/23 ,  G06F16/22

Abstract: The subject technology initiates an online index building process of an index of a hybrid table. The subject technology includes, in the index, a new index record for each record from the hybrid table. The subject technology validates the index. The subject technology retrieves a first metadata version of the hybrid table from a metadata database. The subject technology retrieves a second metadata version of the hybrid table from a distributed data store. The subject technology compares the first metadata version with the second metadata version. The subject technology determines that the second metadata version is less than or equal to the first metadata version. The subject technology, in response to successfully validating the index, indicates in a particular entry of the index that validation of the index has succeeded.

公开(公告)号：US20230325409A1

公开(公告)日：2023-10-12

申请号：US18326683

申请日：2023-05-31

Applicant: Snowflake Inc.

Inventor： Chen Luo ,  Alexander Miller ,  William Waddington ,  Zhaohui Zhang

IPC: G06F16/27 ,  G06F16/23

CPC classification number: G06F16/273 ,  G06F16/2343 ,  G06F16/2379

Abstract: The subject technology performs, using asynchronous threads of execution nodes of a distributed database, asynchronous transformation and compaction of key-value pairs of data that correspond to a plurality of transactional queries, the asynchronous threads transforming the key-value pairs from an initial format to a commit format that includes a commit time of data committed to the distributed database, the asynchronous threads compacting the key-value pairs by deleting the key-value pairs in the initial format. The subject technology processes, using transactional threads in the execution nodes, the plurality of transactional queries to generate results data, the asynchronous threads performing the asynchronous transformation and compaction while the transactional threads generate the results data. The subject technology stores the results data.

公开(公告)号：US20230325376A1

公开(公告)日：2023-10-12

申请号：US18326950

申请日：2023-05-31

Applicant: Snowflake Inc.

Inventor： Alexander Miller ,  William Waddington

IPC: G06F16/22 ,  G06F16/23 ,  G06F16/248 ,  G06F16/2455

CPC classification number: G06F16/2329 ,  G06F16/2282 ,  G06F16/24568 ,  G06F16/248

Abstract: The subject technology performs a read operation from a first transaction on a table in a database. The subject technology determines a first commit version identifier corresponding to first data resulting from the read operation. The subject technology proceeds to perform a particular write operation from the first transaction. The subject technology determines that the first commit version identifier fails to match a second commit version identifier corresponding to a second transaction. The subject technology aborts the particular write operation from the first transaction. The subject technology performs a particular read operation from the first transaction. The subject technology determines a particular commit version identifier corresponding to particular data resulting from the particular read operation. The subject technology retries to perform the particular write operation. The subject technology performs the particular write operation in response to the particular commit version identifier matching the second commit version identifier.

公开(公告)号：US20230195719A1

公开(公告)日：2023-06-22

申请号：US17555197

申请日：2021-12-17

Applicant: Snowflake Inc.

Inventor： Chen Luo ,  Alexander Miller ,  William Waddington ,  Mingze Xu

IPC: G06F16/23 ,  G06F16/27

CPC classification number: G06F16/2379 ,  G06F16/27

Abstract: The subject technology receives a key-value pair. The subject technology determines a value size of the key-value pair is greater than a threshold size for an out of line blob (OLB) key space. The subject technology performs a write operation of a key-pointer pair into a regular space. The subject technology performs a plurality of write operations for multiple key-value parts of the key-value pair into the OLB key space. The subject technology performs conflict checking of the regular space for the key-value pair.

公开(公告)号：US20230055715A1

公开(公告)日：2023-02-23

申请号：US17463353

申请日：2021-08-31

Applicant: Snowflake Inc.

Inventor： Chen Luo ,  Alexander Miller ,  William Waddington ,  Zhaohui Zhang

IPC: G06F16/27 ,  G06F16/23

Abstract: A distributed database compaction system can perform compaction for transactional queries in an asynchronous matter without affecting completion of the queries. The compaction system can implement asynchronous transformation of key pairs in the database, and older keys can be periodically purged using a scheduled compactor. Subsequent queries use the compacted stored keys to perform efficient queries with direct reads of committed transactions and more efficient access to key values stores of the distributed database.

公开(公告)号：US11514064B2

公开(公告)日：2022-11-29

申请号：US17663248

申请日：2022-05-13

Applicant: Snowflake Inc.

Inventor： Jeffrey Rosen ,  Abdul Munir ,  Jiaqi Yan ,  William Waddington ,  Prasanna Rajaperumal ,  Thierry Cruanes

IPC: G06F16/24 ,  G06F16/2458 ,  G06F16/2453 ,  G06F16/2455 ,  G06F9/50

Abstract: Resource provisioning systems and methods are described. In an embodiment, a system includes a plurality of shared storage devices collectively storing database data, an execution platform, and a compute service manager. The compute service manager is configured to determine a task to be executed in response to a trigger event and determine a query plan for executing the task, wherein the query plan comprises a plurality of discrete subtasks. The compute service manager is further configured to assign the plurality of discrete subtasks to one or more nodes of a plurality of nodes of the execution platform, determine whether execution of the task is complete, and in response to determining the execution of the task is complete, store a record in the plurality of shared storage devices indicating the task was completed.

公开(公告)号：US20220269676A1

公开(公告)日：2022-08-25

申请号：US17663248

申请日：2022-05-13

Applicant: Snowflake Inc.

Inventor： Jeffrey Rosen ,  Abdul Munir ,  Jiaqi Yan ,  William Waddington ,  Prasanna Rajaperumal ,  Thierry Cruanes

IPC: G06F16/2458 ,  G06F16/2453 ,  G06F9/50 ,  G06F16/2455

Abstract: Resource provisioning systems and methods are described. In an embodiment, a system includes a plurality of shared storage devices collectively storing database data, an execution platform, and a compute service manager. The compute service manager is configured to determine a task to be executed in response to a trigger event and determine a query plan for executing the task, wherein the query plan comprises a plurality of discrete subtasks. The compute service manager is further configured to assign the plurality of discrete subtasks to one or more nodes of a plurality of nodes of the execution platform, determine whether execution of the task is complete, and in response to determining the execution of the task is complete, store a record in the plurality of shared storage devices indicating the task was completed.

公开(公告)号：US11360989B2

公开(公告)日：2022-06-14

申请号：US17462699

申请日：2021-08-31

Applicant: Snowflake Inc.

Inventor： Jeffrey Rosen ,  Abdul Munir ,  Jiaqi Yan ,  William Waddington ,  Prasanna Rajaperumal ,  Thierry Cruanes

IPC: G06F16/24 ,  G06F16/2458 ,  G06F16/2453 ,  G06F9/50 ,  G06F16/2455

Abstract: Resource provisioning systems and methods are described. In an embodiment, a system includes a plurality of shared storage devices collectively storing database data, an execution platform, and a compute service manager. The compute service manager is configured to determine a task to be executed in response to a trigger event and determine a query plan for executing the task, wherein the query plan comprises a plurality of discrete subtasks. The compute service manager is further configured to assign the plurality of discrete subtasks to one or more nodes of a plurality of nodes of the execution platform, determine whether execution of the task is complete, and in response to determining the execution of the task is complete, store a record in the plurality of shared storage devices indicating the task was completed.

公开(公告)号：US20220092050A1

公开(公告)日：2022-03-24

申请号：US17454368

申请日：2021-11-10

Applicant: Snowflake Inc.

Inventor： Chen Luo ,  Alexander Miller ,  William Waddington ,  Zhaohui Zhang

IPC: G06F16/23

Abstract: The subject technology inserts, by a first transaction, a new version of an object, the inserting corresponding to a first statement from the first transaction, the first statement including information indicating a first key, a first transaction identifier, a first statement number, and a first restart count. The subject technology performs, by a second transaction, a range read, the range read including information indicating the first key. The subject technology receives, after performing the range read, a set of conflicting transactions. The subject technology determines that a conflict occurred between the first transaction and a third transaction from the set of conflicting transactions.