公开(公告)号：US12026153B2

公开(公告)日：2024-07-02

申请号：US18179762

申请日：2023-03-07

Applicant: Snowflake Inc.

Inventor： Torsten Grabs ,  Istvan Cseri ,  Benoit Dageville

IPC: G06F16/00 ,  G06F16/22 ,  G06F16/23 ,  G06F16/2455

CPC classification number: G06F16/2379 ,  G06F16/2282 ,  G06F16/24554 ,  G06F16/2456

Abstract: Systems, methods, and devices for tracking a series of changes to database data are disclosed. A method includes executing a transaction to modify data in a micro-partition of a table of a database by generating a new micro-partition that embodies the transaction. The method includes associating transaction data with the new micro-partition, wherein the transaction data comprises a timestamp when the transaction was fully executed, and further includes associating modification data with the new micro-partition that comprises an indication of one or more rows of the table that were modified by the transaction. The method includes joining the transaction data with the modification data to generate joined data and querying the joined data to determine a listing of intermediate modifications made to the table between a first timestamp and a second timestamp.

公开(公告)号：US20240111762A1

公开(公告)日：2024-04-04

申请号：US18526666

申请日：2023-12-01

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F16/242 ,  G06F3/06 ,  G06F9/54 ,  G06F16/16 ,  G06F16/22 ,  G06F16/23 ,  G06F16/2455 ,  G06F16/25

CPC classification number: G06F16/2423 ,  G06F3/0605 ,  G06F3/0644 ,  G06F3/0653 ,  G06F3/067 ,  G06F9/542 ,  G06F16/164 ,  G06F16/2282 ,  G06F16/2358 ,  G06F16/2393 ,  G06F16/24557 ,  G06F16/256

Abstract: Disclosed herein are systems and methods for efficiently querying external tables. In an embodiment, a database platform receives a query that is directed at least in part to external data in an external table stored on a data storage platform that is external to the database platform. The external table includes a plurality of partitions. The database platform identifies, from external-table metadata, a subset of the plurality of partitions of the external table as including data that potentially satisfies the query. The external-table metadata is stored by the database platform. The database platform identifies data that satisfies the query by scanning the identified subset of the partitions, and responds to the query at least in part with the identified data that satisfies the query.

公开(公告)号：US11507571B2

公开(公告)日：2022-11-22

申请号：US16385720

申请日：2019-04-16

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F16/23 ,  G06F16/248 ,  G06F16/2458 ,  G06F3/06 ,  G06F16/25

Abstract: Systems, methods, and devices for generating a materialized view over an external table. A method includes connecting a database platform to an external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes generating, by the database platform, a materialized view over the external table. The method includes receiving a notification that a modification has been made to the external table, the modification comprising one or more of an addition of a file, a deletion of a file, or an update to a file in a source location for the external table. The method includes, in response to the external table being modified, refreshing the materialized view such that the materialized view comprises an accurate representation of the external table.

公开(公告)号：US20220350795A1

公开(公告)日：2022-11-03

申请号：US17812878

申请日：2022-07-15

Applicant: Snowflake, Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shringte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F16/242 ,  G06F3/06 ,  G06F16/23 ,  G06F16/16 ,  G06F16/25 ,  G06F9/54 ,  G06F16/22 ,  G06F16/2455

Abstract: Disclosed herein are systems and methods for efficiently querying external tables. In an embodiment, a database platform receives a query that is directed at least in part to external data in an external table stored on a data storage platform that is external to the database platform. The external table includes a plurality of partitions. The database platform identifies, from external-table metadata, a subset of the plurality of partitions of the external table as including data that potentially satisfies the query. The external-table metadata is stored by the database platform. The database platform identifies data that satisfies the query by scanning the identified subset of the partitions, and responds to the query at least in part with the identified data that satisfies the query.

公开(公告)号：US20220164339A1

公开(公告)日：2022-05-26

申请号：US17650890

申请日：2022-02-14

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Torsten Grabs ,  Benoit Dageville

IPC: G06F16/23 ,  G06F9/46

Abstract: Systems, methods, and devices for scheduling of data processing tasks are disclosed. A method includes performing a modification on a first set of immutable partitions storing database data to generate a second set of immutable partitions. The second set is associated with a modified version of the database data. A change tracking entry is entered in a stream of tracking entries based on committing the modification. The change tracking entry includes an indication of the modification on the first set of immutable partitions. A stream offset of the stream of tracking entries is advanced based on the entering of the change tracking entry in the stream of tracking entries. The stream offset indicates a timestamp associated with a latest committed modification to the database data. A data processing task is scheduled for execution on the modified version of the database data based on the advancing of the stream offset.

公开(公告)号：US11294882B2

公开(公告)日：2022-04-05

申请号：US17491106

申请日：2021-09-30

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Istvan Cseri ,  Torsten Grabs ,  Benoit Dageville

IPC: G06F16/23 ,  G06F16/245 ,  G06F16/27 ,  G06F9/46 ,  G06F16/18

Abstract: Systems, methods, and devices for transactional processing of change tracking data for a database are discussed. A method includes generating a micro-partition based on execution of a transaction on a table of a database, the micro-partition reflecting changes made to the table by the transaction. A change tracking entry is generated in response to the execution of the transaction. The change tracking entry includes an indication of one or more modifications made to the table by the transaction and an indication of the micro-partition generated based on the execution of the transaction. The change tracking entry is stored in the micro-partition as metadata. At least one existing micro-partition is removed from the table, responsive to storing the change tracking entry.

公开(公告)号：US20210390095A1

公开(公告)日：2021-12-16

申请号：US17462435

申请日：2021-08-31

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F16/23 ,  G06F16/248 ,  G06F16/2458 ,  G06F3/06 ,  G06F16/25

Abstract: The subject technology receives a notification that a modification has been made to an external table, the modification comprising inserting at least one row of new data to the external table, the at least one row corresponding to a first micro-partition that includes a first portion of data from the external table prior to the inserting. The subject technology, in response to the notification indicating the modification to the external table, generates a new micro-partition different from the first micro-partition, the new micro-partition including the inserted at least one row of new data and the first portion of data from the external table. The subject technology generates a refreshed materialized view based at least in part on the generated new micro-partition such that the refreshed materialized view comprises a representation of the external table after the modification has been made.

公开(公告)号：US20210390094A1

公开(公告)日：2021-12-16

申请号：US17461208

申请日：2021-08-30

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Torsten Grabs ,  Benoit Dageville

IPC: G06F16/23 ,  G06F9/46

Abstract: Systems, methods, and devices for executing a task on database data in response to a trigger event are disclosed. A method includes executing a transaction on a table comprising database data, wherein executing the transaction comprises generating a new table version. The method includes, in response to the transaction being fully executed, generating a change tracking entry comprising an indication of one or more modifications made to the table by the transaction and storing the change tracking entry in a change tracking stream. The method includes executing a task on the new table version in response to a trigger event.

公开(公告)号：US20210349886A1

公开(公告)日：2021-11-11

申请号：US17384269

申请日：2021-07-23

Applicant: Snowflake Inc

Inventor： Torsten Grabs ,  Istvan Cseri ,  Benoit Dageville

IPC: G06F16/23 ,  G06F16/22 ,  G06F16/2455

Abstract: Systems, methods, and devices for tracking a series of changes to database data are disclosed. A method includes executing a transaction to modify data in a micro-partition of a table of a database by generating a new micro-partition that embodies the transaction. The method includes associating transaction data with the new micro-partition, wherein the transaction data comprises a timestamp when the transaction was fully executed, and further includes associating modification data with the new micro-partition that comprises an indication of one or more rows of the table that were modified by the transaction. The method includes joining the transaction data with the modification data to generate joined data and querying the joined data to determine a listing of intermediate modifications made to the table between a first timestamp and a second timestamp.

公开(公告)号：US11163757B2

公开(公告)日：2021-11-02

申请号：US17200751

申请日：2021-03-12

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F3/06 ,  G06F16/2453 ,  G06F16/25 ,  G06F16/23 ,  G06F16/242

Abstract: Systems, methods, and devices for querying over an external table are disclosed. A method includes connecting a database platform to an external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes receiving a query comprising a predicate, the query directed at least to data in the external table. The method includes determining, based on metadata, one or more partitions in the external table comprising data satisfying the predicate. The method includes pruning, based on the metadata, all partitions in the external table that do not comprise any data satisfying the predicate. The method includes generating a query plan comprising a plurality of discrete subtasks. The method includes assigning, based on the metadata, the plurality of discrete subtasks to one or more nodes in an execution platform.