公开(公告)号：US11347485B1

公开(公告)日：2022-05-31

申请号：US17389937

申请日：2021-07-30

Applicant: Snowflake Inc.

Inventor： Elliott Brossard ,  Istvan Cseri ,  Derek Denny-Brown ,  Filip Drozdowski ,  Isaac Kunen ,  Edward Ma

IPC: G06F9/44 ,  G06F8/41 ,  G06F16/22 ,  G06F8/30

Abstract: A system comprises at least one hardware processor and a memory storing instructions. When executed, the instructions cause the at least one hardware processor to perform operations comprising receiving, in a compiling process, a request to create a Java user-defined table function (Java UDTF), the Java UDTF including code related to receiving one or more input tables and transforming the one or more input tables to an output table; verifying a construct of the Java UDTF in the request is correct; and compiling to generate execution code that includes the Java UDTF when the construct of the Java UDTF is correct.

公开(公告)号：US20220100758A1

公开(公告)日：2022-03-31

申请号：US17039473

申请日：2020-09-30

Applicant: Snowflake Inc.

Inventor： Elliott Brossard ,  Istvan Cseri ,  Isaac Kunen ,  Nitya Kumar Sharma ,  Igor Zinkovsky

IPC: G06F16/2453

Abstract: External functions can provide users of a database system the ability to invoke external services while operating from inside the database system. The techniques described herein address may detect throttling by the external system and automatically adjust or modify a request rate accordingly. Also, the techniques described herein may increase the request rate to optimize the use of the resources provided by the external system. These techniques allow the database system to improve efficiency when working with external services, which may or may not be elastic or scalable. Moreover, these techniques can decrease query execution times and reduce costs.

公开(公告)号：US20220075776A1

公开(公告)日：2022-03-10

申请号：US17455798

申请日：2021-11-19

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F16/242 ,  G06F3/06 ,  G06F16/23 ,  G06F16/16 ,  G06F16/25 ,  G06F9/54 ,  G06F16/22

Abstract: Disclosed herein are systems and methods for pruning external data. In an embodiment, a database platform receives a query directed at least in part to external data in an external table on an external data storage platform. The external table includes partitions that correspond to storage locations in a source directory of the external data storage platform. The storage locations contain files that contain the external data. The database platform identifies, from external-table metadata that is stored by the database platform and that maps the partitions of the external table to the storage locations in the source directory, a subset of the partitions as including data that potentially satisfies the query. The database platform identifies data that satisfies the query by scanning the identified subset of the partitions, and responds to the query at least in part with the identified data that satisfies the query.

公开(公告)号：US11269868B2

公开(公告)日：2022-03-08

申请号：US17219854

申请日：2021-03-31

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F16/00 ,  G06F16/242 ,  G06F3/06 ,  G06F16/23 ,  G06F16/16 ,  G06F16/25 ,  G06F9/54 ,  G06F16/22

Abstract: Systems, methods, and devices for automated maintenance of external tables in database systems are disclosed. A method includes receiving, by a database platform, read access to content in an external data storage platform that is separate from the database platform. The method includes defining an external table based on the content in the external data storage platform. The method includes connecting the database platform to the external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes generating metadata for the external table, the metadata comprising information about data stored in the external table. The method includes receiving a notification that a modification has been made to the content in the external data storage platform, the modification comprising one or more of an addition of a file, a deletion of a file, or an update to a file in a source location for the external table. The method includes refreshing the metadata for the external table in response to the modification being made to the content in the external data storage platform.

公开(公告)号：US20220058160A1

公开(公告)日：2022-02-24

申请号：US17517398

申请日：2021-11-02

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Tyler Jones

IPC: G06F16/17 ,  G06F16/182 ,  G06F16/18 ,  G06F11/14 ,  G06F16/14

Abstract: The subject technology receives an operation to perform on an external data source accessible via a network, the external data source being hosted by an external system separate from a network-based database system. The subject technology determines a set of shards corresponding to the external data source. The subject technology determines a set of offsets of each shard of the set of shards. The subject technology, based on the set of shards and the set of offsets, performs the operation on the external data source. The subject technology provides an indication that the operation is complete.

公开(公告)号：US11216481B2

公开(公告)日：2022-01-04

申请号：US17345484

申请日：2021-06-11

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Qiming Jiang

IPC: G06F16/25 ,  G06N20/00 ,  G06F9/46

Abstract: The subject technology determines a number of execution nodes to execute a task. The subject technology determines that no virtual warehouse from a pool of virtual warehouses includes at least the number of execution nodes. The subject technology generates a new virtual warehouse including at least the number of execution nodes. The subject technology determines whether a previous execution of a prior task was completed in less than a threshold time period of a time interval, the time interval corresponding to a particular period of time for executing at least one task. The subject technology increments a score corresponding to a size of a particular virtual warehouse. The subject technology selects the new virtual warehouse from the pool of virtual warehouses based at least in part on the incremented score and the number of execution nodes included in the selected new virtual warehouse.

公开(公告)号：US11138192B1

公开(公告)日：2021-10-05

申请号：US17302388

申请日：2021-04-30

Applicant: Snowflake Inc.

Inventor： Srilakshmi Chintala ,  Istvan Cseri ,  Albert L. Hu ,  Isaac Kunen ,  Nitya Kumar Sharma ,  Igor Zinkovsky

IPC: G06F16/00 ,  G06F16/245 ,  G06F16/22 ,  H04L29/06 ,  G06F16/958

Abstract: A query referencing an external table function provided by a remote software component is received. Requests to execute the external table function on input data are sent to a proxy service. A first request includes a batch of input rows from the input data. A first response to the first request received from the proxy service includes a first portion of result data and a pagination token. The pagination token indicates that at least a second portion of the result data corresponding to the first batch of input rows is to be obtained from the remote software component. Based on the pagination token, a second request is sent to obtain the second portion of the result data. One or more responses are received from the proxy service that comprise at least the second portion of the result data. The result data is processed according to the query.

公开(公告)号：US11126484B2

公开(公告)日：2021-09-21

申请号：US17238558

申请日：2021-04-23

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Isaac Kunen ,  Igor Zinkovsky

IPC: G06F9/44 ,  G06F9/54 ,  H04L29/06 ,  G06F16/242 ,  H04L29/08

Abstract: A query referencing a function associated with a remote software component is received by a network-based data warehouse system. Temporary security credentials corresponding to a role at a cloud computing service platform are obtained. The role has permission to send calls to a web endpoint corresponding to the remote software component. A request comprising input data and electronically signed using the temporary security credentials is sent to a web Application Programming Interface (API) management system of the cloud computing service platform. The request, when received by the web API management system, causes the web API management system to invoke external functionality provided by the remote software component at the web endpoint with respect to the input data. A response comprising a result of invoking the external functionality is received from the web API management system, and the result data is processed according to the query.

公开(公告)号：US20210279209A1

公开(公告)日：2021-09-09

申请号：US16813048

申请日：2020-03-09

Applicant: Snowflake Inc.

Inventor： Istvan Cseri ,  Tyler Jones

IPC: G06F16/17 ,  G06F16/182 ,  G06F16/14 ,  G06F16/18 ,  G06F11/14

Abstract: The subject technology receives an operation to perform on an external data source accessible via a network, the external data source being hosted by an external system separate from a network-based database system. The subject technology determines a set of shards corresponding to the external data source. The subject technology determines a set of offsets of each shard of the set of shards. The subject technology, based on the set of shards and the set of offsets, performs the operation on the external data source. The subject technology provides an indication that the operation is complete.

公开(公告)号：US11106661B2

公开(公告)日：2021-08-31

申请号：US17219600

申请日：2021-03-31

Applicant: Snowflake Inc.

Inventor： Torsten Grabs ,  Istvan Cseri ,  Benoit Dageville

IPC: G06F17/00 ,  G06F16/23 ,  G06F16/22 ,  G06F16/2455

Abstract: Systems, methods, and devices for tracking a series of changes to database data are disclosed. A method includes executing a transaction to modify data in a micro-partition of a table of a database by generating a new micro-partition that embodies the transaction. The method includes associating transaction data with the new micro-partition, wherein the transaction data comprises a timestamp when the transaction was fully executed, and further includes associating modification data with the new micro-partition that comprises an indication of one or more rows of the table that were modified by the transaction. The method includes joining the transaction data with the modification data to generate joined data and querying the joined data to determine a listing of intermediate modifications made to the table between a first timestamp and a second timestamp.