公开(公告)号：US12026153B2

公开(公告)日：2024-07-02

申请号：US18179762

申请日：2023-03-07

申请人： Snowflake Inc.

发明人： Torsten Grabs ,  Istvan Cseri ,  Benoit Dageville

IPC分类号： G06F16/00 ,  G06F16/22 ,  G06F16/23 ,  G06F16/2455

CPC分类号： G06F16/2379 ,  G06F16/2282 ,  G06F16/24554 ,  G06F16/2456

摘要： Systems, methods, and devices for tracking a series of changes to database data are disclosed. A method includes executing a transaction to modify data in a micro-partition of a table of a database by generating a new micro-partition that embodies the transaction. The method includes associating transaction data with the new micro-partition, wherein the transaction data comprises a timestamp when the transaction was fully executed, and further includes associating modification data with the new micro-partition that comprises an indication of one or more rows of the table that were modified by the transaction. The method includes joining the transaction data with the modification data to generate joined data and querying the joined data to determine a listing of intermediate modifications made to the table between a first timestamp and a second timestamp.

公开(公告)号：US12007961B2

公开(公告)日：2024-06-11

申请号：US18345987

申请日：2023-06-30

申请人： Snowflake Inc.

发明人： Istvan Cseri ,  Benoit Dageville ,  Ganeshan Ramachandran Iyer ,  Yucan Liu ,  Jiaqi Yan

IPC分类号： G06F16/00 ,  G06F16/21

CPC分类号： G06F16/211

摘要： Techniques for schema mismatch detection and evolution are described. When data is being uploaded into a source table, schema of the data to be uploaded can be compared with the schema for the source table. If a schema mismatch is detected, the schema of the source table can be modified, and the upload can be continued without data loss.

公开(公告)号：US20240126765A1

公开(公告)日：2024-04-18

申请号：US18392327

申请日：2023-12-21

申请人： Snowflake Inc.

发明人： Tyler Arthur Akidau ,  Istvan Cseri ,  Tyler Jones ,  Daniel E. Sotolongo ,  Zhuo Zhang

IPC分类号： G06F16/2455 ,  G06F16/22 ,  G06F16/2453 ,  G06F16/25

CPC分类号： G06F16/24568 ,  G06F16/2219 ,  G06F16/24544 ,  G06F16/2456 ,  G06F16/258

摘要： A streaming ingest platform can improve latency and expense issues related to uploading data into a cloud data system. The streaming ingest platform can organize the data to be ingested into per-table chunks and per-account blobs. This data may be committed and may be made available for query processing before it is ingested into the target source tables. This significantly improves latency issues. The streaming ingest platform can also accommodate uploading data from various sources with different processing and communication capabilities, such as Internet of Things (IOT) devices.

公开(公告)号：US20240111762A1

公开(公告)日：2024-04-04

申请号：US18526666

申请日：2023-12-01

申请人： Snowflake Inc.

发明人： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC分类号： G06F16/242 ,  G06F3/06 ,  G06F9/54 ,  G06F16/16 ,  G06F16/22 ,  G06F16/23 ,  G06F16/2455 ,  G06F16/25

CPC分类号： G06F16/2423 ,  G06F3/0605 ,  G06F3/0644 ,  G06F3/0653 ,  G06F3/067 ,  G06F9/542 ,  G06F16/164 ,  G06F16/2282 ,  G06F16/2358 ,  G06F16/2393 ,  G06F16/24557 ,  G06F16/256

摘要： Disclosed herein are systems and methods for efficiently querying external tables. In an embodiment, a database platform receives a query that is directed at least in part to external data in an external table stored on a data storage platform that is external to the database platform. The external table includes a plurality of partitions. The database platform identifies, from external-table metadata, a subset of the plurality of partitions of the external table as including data that potentially satisfies the query. The external-table metadata is stored by the database platform. The database platform identifies data that satisfies the query by scanning the identified subset of the partitions, and responds to the query at least in part with the identified data that satisfies the query.

公开(公告)号：US11893029B2

公开(公告)日：2024-02-06

申请号：US18049325

申请日：2022-10-25

申请人： Snowflake Inc.

发明人： Tyler Arthur Akidau ,  Istvan Cseri ,  Tyler Jones ,  Daniel E. Sotolongo ,  Zhuo Zhang

IPC分类号： G06F16/245 ,  G06F16/25 ,  G06F16/22 ,  G06F16/2455 ,  G06F16/2453

CPC分类号： G06F16/24568 ,  G06F16/2219 ,  G06F16/2456 ,  G06F16/24544 ,  G06F16/258

摘要： A streaming ingest platform can improve latency and expense issues related to uploading data into a cloud data system. The streaming ingest platform can organize the data to be ingested into per-table chunks and per-account blobs. This data may be committed and may be made available for query processing before it is ingested into the target source tables. This significantly improves latency issues. The streaming ingest platform can also accommodate uploading data from various sources with different processing and communication capabilities, such as Internet of Things (IOT) devices.

公开(公告)号：US11748327B2

公开(公告)日：2023-09-05

申请号：US17648622

申请日：2022-01-21

申请人： Snowflake Inc.

发明人： Tyler Arthur Akidau ,  Istvan Cseri ,  Tyler Jones ,  Daniel E. Sotolongo

IPC分类号： G06F16/22 ,  G06F16/23 ,  G06F16/2455

CPC分类号： G06F16/2282 ,  G06F16/2358 ,  G06F16/24568

摘要： A system or persistent table may be generated storing changelog information of a primary base table. The system table may then be used to create streams of relevant information. In some examples, the streams may read from the system table for information past a retention period of the primary table while reading from the primary table information in the retention period.

公开(公告)号：US11734301B1

公开(公告)日：2023-08-22

申请号：US17656116

申请日：2022-03-23

申请人： Snowflake Inc.

发明人： Istvan Cseri ,  Tyler Jones ,  Daniel E. Sotolongo ,  Boyuan Zhang

IPC分类号： G06F16/30 ,  G06F16/27

CPC分类号： G06F16/27

摘要： Techniques described herein can enable stream replication. A first deployment can store a table including one or more streams. The techniques described herein can be used to replicate the table at a second deployment while replicating the one or more streams associated with the table. Select prior table versions and partitions in the table are copied to the second deployment to enable stream replication.

公开(公告)号：US11507571B2

公开(公告)日：2022-11-22

申请号：US16385720

申请日：2019-04-16

申请人： Snowflake Inc.

发明人： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC分类号： G06F16/23 ,  G06F16/248 ,  G06F16/2458 ,  G06F3/06 ,  G06F16/25

摘要： Systems, methods, and devices for generating a materialized view over an external table. A method includes connecting a database platform to an external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes generating, by the database platform, a materialized view over the external table. The method includes receiving a notification that a modification has been made to the external table, the modification comprising one or more of an addition of a file, a deletion of a file, or an update to a file in a source location for the external table. The method includes, in response to the external table being modified, refreshing the materialized view such that the materialized view comprises an accurate representation of the external table.

公开(公告)号：US20220350795A1

公开(公告)日：2022-11-03

申请号：US17812878

申请日：2022-07-15

申请人： Snowflake, Inc.

发明人： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shringte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC分类号： G06F16/242 ,  G06F3/06 ,  G06F16/23 ,  G06F16/16 ,  G06F16/25 ,  G06F9/54 ,  G06F16/22 ,  G06F16/2455

摘要： Disclosed herein are systems and methods for efficiently querying external tables. In an embodiment, a database platform receives a query that is directed at least in part to external data in an external table stored on a data storage platform that is external to the database platform. The external table includes a plurality of partitions. The database platform identifies, from external-table metadata, a subset of the plurality of partitions of the external table as including data that potentially satisfies the query. The external-table metadata is stored by the database platform. The database platform identifies data that satisfies the query by scanning the identified subset of the partitions, and responds to the query at least in part with the identified data that satisfies the query.

公开(公告)号：US20220164339A1

公开(公告)日：2022-05-26

申请号：US17650890

申请日：2022-02-14

申请人： Snowflake Inc.

发明人： Istvan Cseri ,  Torsten Grabs ,  Benoit Dageville

IPC分类号： G06F16/23 ,  G06F9/46

摘要： Systems, methods, and devices for scheduling of data processing tasks are disclosed. A method includes performing a modification on a first set of immutable partitions storing database data to generate a second set of immutable partitions. The second set is associated with a modified version of the database data. A change tracking entry is entered in a stream of tracking entries based on committing the modification. The change tracking entry includes an indication of the modification on the first set of immutable partitions. A stream offset of the stream of tracking entries is advanced based on the entering of the change tracking entry in the stream of tracking entries. The stream offset indicates a timestamp associated with a latest committed modification to the database data. A data processing task is scheduled for execution on the modified version of the database data based on the advancing of the stream offset.