公开(公告)号：US20240095229A1

公开(公告)日：2024-03-21

申请号：US18520845

申请日：2023-11-28

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Torsten Grabs ,  Benoit Dageville

IPC: G06F16/23 ,  G06F16/2455 ,  G06F16/27

CPC classification number: G06F16/2358 ,  G06F16/2455 ,  G06F16/278

Abstract: A method includes detecting, by at least one hardware processor, a change request for a table of a database, the table comprising a plurality of micro-partitions. A transaction associated with the change request is executed at a first timestamp. The transaction causes replacement of a first micro-partition of the plurality of micro-partitions with a second micro-partition. A change tracking column is generated in the second micro-partition. The change tracking column comprises metadata for the transaction. A delta for the table between the first timestamp and a second timestamp is generated using the metadata in the change tracking column. The delta indicates changes made to one or more rows of the table between the first time stamp and the second timestamp.

公开(公告)号：US11841849B2

公开(公告)日：2023-12-12

申请号：US17812878

申请日：2022-07-15

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F16/20 ,  G06F16/242 ,  G06F3/06 ,  G06F16/23 ,  G06F16/16 ,  G06F16/25 ,  G06F9/54 ,  G06F16/22 ,  G06F16/2455

CPC classification number: G06F16/2423 ,  G06F3/0605 ,  G06F3/067 ,  G06F3/0644 ,  G06F3/0653 ,  G06F9/542 ,  G06F16/164 ,  G06F16/2282 ,  G06F16/2358 ,  G06F16/2393 ,  G06F16/24557 ,  G06F16/256

Abstract: In an embodiment, a database platform receives a query that is directed at least in part to external data in an external table stored on a data storage platform that is external to the database platform. The external table includes a plurality of partitions. The database platform identifies, from external-table metadata, a subset of the plurality of partitions of the external table as including data that potentially satisfies the query. The external-table metadata is stored by the database platform. The database platform identifies data that satisfies the query by scanning the identified subset of the partitions, and responds to the query at least in part with the identified data that satisfies the query.

公开(公告)号：US11354316B2

公开(公告)日：2022-06-07

申请号：US17561222

申请日：2021-12-23

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F16/00 ,  G06F16/2455 ,  G06F16/23 ,  G06F16/22

Abstract: Disclosed herein are systems and methods for selective scanning of external partitions. In an embodiment, a database platform receives a query directed at least in part to an external table stored on an external data storage platform. The external table is partitioned into partitions corresponding to storage locations in the external data storage platform. The database platform prunes, using external-table metadata that is stored by the database platform and that maps the partitions of the external table to the storage locations in the external data storage platform, those partitions that do not potentially contain data that satisfies the query. The database platform identifies data that satisfies the query by scanning any one or more of the partitions of the external table that were not pruned, and responds to the query at least in part with the identified data that satisfies the query.

公开(公告)号：US11308078B2

公开(公告)日：2022-04-19

申请号：US17389234

申请日：2021-07-29

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Torsten Grabs ,  Benoit Dageville

IPC: G06F16/23 ,  G06F9/46

Abstract: Systems, methods, and devices for executing a task on database data in response to a trigger event are disclosed. A method includes executing a transaction on a table comprising database data, wherein executing the transaction comprises generating a new table version. The method includes, in response to the transaction being fully executed, generating a change tracking entry comprising an indication of one or more modifications made to the table by the transaction and storing the change tracking entry in a change tracking stream. The method includes executing a task on the new table version in response to a trigger event.

公开(公告)号：US20220114180A1

公开(公告)日：2022-04-14

申请号：US17561222

申请日：2021-12-23

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F16/2455 ,  G06F16/22 ,  G06F16/23

Abstract: Disclosed herein are systems and methods for selective scanning of external partitions. In an embodiment, a database platform receives a query directed at least in part to an external table stored on an external data storage platform. The external table is partitioned into partitions corresponding to storage locations in the external data storage platform. The database platform prunes, using external-table metadata that is stored by the database platform and that maps the partitions of the external table to the storage locations in the external data storage platform, those partitions that do not potentially contain data that satisfies the query. The database platform identifies data that satisfies the query by scanning any one or more of the partitions of the external table that were not pruned, and responds to the query at least in part with the identified data that satisfies the query.

公开(公告)号：US20220027368A1

公开(公告)日：2022-01-27

申请号：US17498382

申请日：2021-10-11

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F16/2453

Abstract: Disclosed herein are systems and methods for processing queries over external tables. In an embodiment, a database platform receives a query directed at least to data in an external table stored in a storage platform that is external to the database platform. The database platform uses metadata that summarizes the data in the external table to identify one or more partitions of the external table as potentially including data satisfying the query, and generates a query plan that includes a plurality of discrete subtasks that collectively include instructions to scan the identified one or more partitions of the external table for data satisfying the query. The database platform assigns, based on the metadata, the plurality of discrete subtasks to one or more nodes in an execution platform, and refreshes the metadata in response to a threshold number of modifications being made to the external table.

公开(公告)号：US20220019570A1

公开(公告)日：2022-01-20

申请号：US17491106

申请日：2021-09-30

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Istvan Cseri ,  Torsten Grabs ,  Benoit Dageville

IPC: G06F16/23 ,  G06F16/27 ,  G06F16/245 ,  G06F16/18

Abstract: Systems, methods, and devices for transactional processing of change tracking data for a database are discussed. A method includes generating a micro-partition based on execution of a transaction on a table of a database, the micro-partition reflecting changes made to the table by the transaction. A change tracking entry is generated in response to the execution of the transaction. The change tracking entry includes an indication of one or more modifications made to the table by the transaction and an indication of the micro-partition generated based on the execution of the transaction. The change tracking entry is stored in the micro-partition as metadata. At least one existing micro-partition is removed from the table, responsive to storing the change tracking entry.

公开(公告)号：US11194795B2

公开(公告)日：2021-12-07

申请号：US16385837

申请日：2019-04-16

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F16/20 ,  G06F16/242 ,  G06F3/06 ,  G06F16/23 ,  G06F16/16 ,  G06F16/25 ,  G06F9/54 ,  G06F16/22

Abstract: Systems, methods, and devices for automated maintenance of external tables in database systems are disclosed. A method includes receiving, by a database platform, read access to content in an external data storage platform that is separate from the database platform. The method includes defining an external table based on the content in the external data storage platform. The method includes connecting the database platform to the external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes generating metadata for the external table, the metadata comprising information about data stored in the external table. The method includes receiving a notification that a modification has been made to the content in the external data storage platform, the modification comprising one or more of an addition of a file, a deletion of a file, or an update to a file in a source location for the external table. The method includes refreshing the metadata for the external table in response to the modification being made to the content in the external data storage platform.

公开(公告)号：US20210216541A1

公开(公告)日：2021-07-15

申请号：US17219854

申请日：2021-03-31

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F16/242 ,  G06F3/06 ,  G06F16/23 ,  G06F16/16 ,  G06F16/25 ,  G06F9/54 ,  G06F16/22

Abstract: Systems, methods, and devices for automated maintenance of external tables in database systems are disclosed. A method includes receiving, by a database platform, read access to content in an external data storage platform that is separate from the database platform. The method includes defining an external table based on the content in the external data storage platform. The method includes connecting the database platform to the external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes generating metadata for the external table, the metadata comprising information about data stored in the external table. The method includes receiving a notification that a modification has been made to the content in the external data storage platform, the modification comprising one or more of an addition of a file, a deletion of a file, or an update to a file in a source location for the external table. The method includes refreshing the metadata for the external table in response to the modification being made to the content in the external data storage platform.

公开(公告)号：US10997152B2

公开(公告)日：2021-05-04

申请号：US16944152

申请日：2020-07-31

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Torsten Grabs ,  Benoit Dageville

IPC: G06F16/23 ,  G06F16/27 ,  G06F16/2455

Abstract: The subject technology executes a change on an existing micro-partition of a table, the executing of the change comprising generating a new micro-partition that embodies the change and deleting the existing micro-partition. The subject technology receives a request for a delta for the table between a first timestamp and a second timestamp. The subject technology responsively determines the delta based on a table history, the delta including information indicating at least one operation that was performed to at least one row of a set of rows of the table, without including information as to intermediate changes made to at least one row of the set of rows of the table between the first timestamp and the second timestamp, that facilitates a reduction in storage of historical versions of the set of rows of the table.