公开(公告)号：US11899646B2

公开(公告)日：2024-02-13

申请号：US18193069

申请日：2023-03-30

Applicant: Snowflake Inc.

Inventor： Selcuk Aya ,  Thierry Cruanes ,  Istvan Cseri ,  Benoit Dageville ,  Marcia Feitel ,  Steven P. Herbert ,  Xinglian Liu ,  James Malone ,  Subramanian Muralidhar ,  Muthunagappan Muthuraman ,  Polita Paulus ,  Marianne Shaw ,  Nileema Shingte ,  Wai Sing Wong ,  Jiaqi Yan

IPC: G06F16/22 ,  G06F16/25 ,  G06F16/23

CPC classification number: G06F16/2282 ,  G06F16/2379 ,  G06F16/258

Abstract: The subject technology receives a command to commit a table in a different table format on an external volume. The subject technology generates a first snapshot of the table on internal storage. The subject technology generates a first list of metadata files on the internal storage. The subject technology generates, based on the first list of metadata files, a first set of metadata files on the internal storage. The subject technology generates a second snapshot of the table on the external volume. The subject technology generates a second list of metadata files on the external volume. The subject technology generates, based on the second list of metadata files, a second set of metadata files on the external volume. The subject technology generates a first set of data files in a different file format on the external volume.

公开(公告)号：US11860850B2

公开(公告)日：2024-01-02

申请号：US18048912

申请日：2022-10-24

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Tyler Jones ,  Daniel Mills ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F16/2455 ,  G06F16/27 ,  G06F16/22

CPC classification number: G06F16/2358 ,  G06F16/2282 ,  G06F16/2456 ,  G06F16/24568 ,  G06F16/278

Abstract: Provided herein are systems and methods for a stream object configuration, including configuring and processing streams on views and streams on tables. For example, a method includes parsing, by at least one hardware processor, a received query to determine a plurality of data processing operations associated with a base table. A stream object on the base table is generated. The stream object includes a plurality of query results from executing the plurality of data processing operations and a corresponding plurality of time stamps of times the plurality of query results are obtained. A stream expansion of the stream object is performed. The stream expansion is based on computing changes to the base table from the plurality of data processing operations.

公开(公告)号：US20230401193A1

公开(公告)日：2023-12-14

申请号：US18193069

申请日：2023-03-30

Applicant: Snowflake Inc.

Inventor： Selcuk Aya ,  Thierry Cruanes ,  Istvan Cseri ,  Benoit Dageville ,  Marcia Feitel ,  Steven P. Herbert ,  Xinglian Liu ,  James Malone ,  Subramanian Muralidhar ,  Muthunagappan Muthuraman ,  Polita Paulus ,  Marianne Shaw ,  Nileema Shingte ,  Wai Sing Wong ,  Jiaqi Yan

IPC: G06F16/22 ,  G06F16/23 ,  G06F16/25

CPC classification number: G06F16/2282 ,  G06F16/2379 ,  G06F16/258

Abstract: The subject technology receives a command to commit a table in a different table format on an external volume. The subject technology generates a first snapshot of the table on internal storage. The subject technology generates a first list of metadata files on the internal storage. The subject technology generates, based on the first list of metadata files, a first set of metadata files on the internal storage. The subject technology generates a second snapshot of the table on the external volume. The subject technology generates a second list of metadata files on the external volume. The subject technology generates, based on the second list of metadata files, a second set of metadata files on the external volume. The subject technology generates a first set of data files in a different file format on the external volume.

公开(公告)号：US11762838B2

公开(公告)日：2023-09-19

申请号：US18162513

申请日：2023-01-31

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Istvan Cseri ,  Torsten Grabs ,  Benoit Dageville

IPC: G06F16/23 ,  G06F16/18 ,  G06F16/245 ,  G06F16/27 ,  G06F9/46

CPC classification number: G06F16/2358 ,  G06F16/1865 ,  G06F16/2322 ,  G06F16/2365 ,  G06F16/245 ,  G06F16/278 ,  G06F9/466

Abstract: A method for table data processing includes retrieving, by at least one hardware processor, first table data associated with a first table version from a plurality of partitions of a storage device. The method further includes detecting, by the at least one hardware processor, an executed transaction causing a modification to the first table data resulting in a second table data associated with a second table version. The second table data is modified to include partition metadata. The partition metadata includes table version information with a mapping of the first table version and the second table version to a plurality of transactions associated with generating the first table data and the second table data.

公开(公告)号：US11748318B1

公开(公告)日：2023-09-05

申请号：US18104253

申请日：2023-01-31

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Benoit Dageville ,  Ganeshan Ramachandran Iyer ,  Yucan Liu ,  Jiaqi Yan

IPC: G06F16/00 ,  G06F16/21

CPC classification number: G06F16/211

Abstract: Techniques for schema mismatch detection and evolution are described. When data is being uploaded into a source table, schema of the data to be uploaded can be compared with the schema for the source table. If a schema mismatch is detected, the schema of the source table can be modified, and the upload can be continued without data loss.

公开(公告)号：US11675784B2

公开(公告)日：2023-06-13

申请号：US17823132

申请日：2022-08-30

Applicant: Snowflake Inc.

Inventor： Srilakshmi Chintala ,  Istvan Cseri ,  Albert L. Hu ,  Isaac Kunen ,  Nitya Kumar Sharma ,  Igor Zinkovsky

IPC: G06F16/22 ,  G06F16/2458 ,  G06F16/245 ,  G06F16/958 ,  H04L9/40

CPC classification number: G06F16/245 ,  G06F16/2282 ,  G06F16/2471 ,  G06F16/986 ,  H04L63/0281 ,  H04L63/0846

Abstract: A query referencing an external table function provided by a remote software component is received. Requests to execute the external table function on input data are sent to a proxy service. A first request includes a batch of input rows from the input data. A first response to the first request received from the proxy service includes a first portion of result data and a pagination token. The pagination token indicates that at least a second portion of the result data corresponding to the first batch of input rows is to be obtained from the remote software component. Based on the pagination token, a second request is sent to obtain the second portion of the result data. One or more responses are received from the proxy service that comprise at least the second portion of the result data. The result data is processed according to the query.

公开(公告)号：US11645262B2

公开(公告)日：2023-05-09

申请号：US17162790

申请日：2021-01-29

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Torsten Grabs ,  Benoit Dageville

IPC: G06F16/23 ,  G06F16/27 ,  G06F16/2455

CPC classification number: G06F16/2358 ,  G06F16/2455 ,  G06F16/278

Abstract: The subject technology executes a change on an existing micro-partition of a table of a database, the executing of the change comprising generating a new micro-partition that embodies the change. The subject technology receives a request for a delta for the table between a first timestamp and a second timestamp. The subject technology queries at least one change tracking column to determine the delta between the first timestamp and the second timestamp, the delta including information indicating at least one database operation that was performed to at least one row of a set of rows of the table, without including information as to intermediate changes made to at least one row of the set of rows of the table between the first timestamp and the second timestamp, that facilitates a reduction in storage of historical versions of the set of rows of the table.

公开(公告)号：US20230092367A1

公开(公告)日：2023-03-23

申请号：US18048912

申请日：2022-10-24

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Tyler Jones ,  Daniel Mills ,  Daniel E. Sotolongo

IPC: G06F16/23 ,  G06F16/2455 ,  G06F16/27 ,  G06F16/22

Abstract: Provided herein are systems and methods for a stream object configuration, including configuring and processing streams on views and streams on tables. For example, a method includes parsing, by at least one hardware processor, a received query to determine a plurality of data processing operations associated with a base table. A stream object on the base table is generated. The stream object includes a plurality of query results from executing the plurality of data processing operations and a corresponding plurality of time stamps of times the plurality of query results are obtained. A stream expansion of the stream object is performed. The stream expansion is based on computing changes to the base table from the plurality of data processing operations.

公开(公告)号：US11593310B2

公开(公告)日：2023-02-28

申请号：US17821985

申请日：2022-08-24

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Tyler Jones

IPC: G06F16/17 ,  G06F16/182 ,  G06F16/18 ,  G06F11/14 ,  G06F16/14

Abstract: The subject technology determines, using a connection to an external data source, a set of shards stored in an external data source, the connection to the external data source being established using an external integration, the external integration including security and configuration information. The subject technology determines a set of offsets of each shard of the set of shards. The subject technology generates a query plan indicating a degree of parallelism based at least in part on a size of the set of offsets. The subject technology, based on the set of shards and the set of offsets, performs an operation on the external data source by performing, using the connection to the external data source, a write operation from a query statement on the external data source, the external data source being different than a storage platform associated with the system.

公开(公告)号：US11526517B2

公开(公告)日：2022-12-13

申请号：US17647500

申请日：2022-01-10

Applicant: Snowflake Inc.

Inventor： Tyler Arthur Akidau ,  Istvan Cseri ,  Tyler Jones ,  Daniel E. Sotolongo ,  Zhuo Zhang

IPC: G06F16/245 ,  G06F16/2455 ,  G06F16/2453

Abstract: A streaming ingest platform can improve latency and expense issues related to uploading data into a cloud data system. The streaming ingest platform can organize the data to be ingested into per-table chunks and per-account blobs. This data may be committed and may be made available for query processing before it is ingested into the target source tables. This significantly improves latency issues. The streaming ingest platform can also accommodate uploading data from various sources with different processing and communication capabilities, such as Internet of Things (IOT) devices.