公开(公告)号：US11423081B1

公开(公告)日：2022-08-23

申请号：US17390969

申请日：2021-08-01

Applicant: Snowflake Inc.

Inventor： Elliott Brossard ,  Sukruth Komarla Sukumar ,  Isaac Kunen ,  Ju-Yi Kuo ,  Jonathan Leang ,  Edward Ma ,  Schuyler James Manchester ,  Polita Paulus ,  Saurin Shah ,  Igor Zinkovsky

IPC: G06F16/00 ,  G06F16/901 ,  G06F16/955 ,  G06F16/2455 ,  G06F16/22 ,  G06F16/908

Abstract: A file access system for user defined functions (UDFs) can be implemented on a distributed database system. The system can store UDF signatures and interfaces (e.g., classes, sub-classes) that can be called by other users. Upon a UDF being called, one or more interface objects (e.g., InputStream) can be created and requests transferred to a execution node via a network channel. The execution node can implement multiple threads that are authorized and download file data from a staging location (e.g., internal stage, external stage) concurrently.

公开(公告)号：US20220237231A1

公开(公告)日：2022-07-28

申请号：US17657548

申请日：2022-03-31

Applicant: Snowflake Inc.

Inventor： Elliott Brossard ,  Sukruth Komarla Sukumar ,  Isaac Kunen ,  Ju-yi Kuo ,  Jonathan Lee Leang ,  Edward Ma ,  Schuyler James Manchester ,  Polita Paulus ,  Saurin Shah ,  Igor Zinkovsky

IPC: G06F16/901 ,  G06F16/955 ,  G06F16/2455 ,  G06F16/22 ,  G06F16/908

Abstract: A file access system for user defined functions (UDFs) can be implemented on a distributed database system. The system can store UDF interfaces and file reference objects that can be called by other users. Upon a UDF being called, files on a stage, one or more interface objects (e.g., InputStream), and file reference objects can be implemented by execution nodes of the distributed database system. The execution nodes can implement multiple threads that are authenticated and can download file data from a staging location concurrently.

公开(公告)号：US20210406310A1

公开(公告)日：2021-12-30

申请号：US17244349

申请日：2021-04-29

Applicant: Snowflake Inc.

Inventor： Polita Paulus ,  Aravind Ramarathinam ,  Saurin Shah ,  Sukruth Komarla Sukumar

IPC: G06F16/901 ,  G06F16/908 ,  G06F16/2455 ,  G06F16/955 ,  G06F16/22

Abstract: Disclosed herein are systems and methods for implementing a file-catalog table for a file stage. In an embodiment, a data platform receives, from a client, a request that the data platform create a file-catalog table for a file stage of a user account associated with the client, where the file stage contains one or more files at a storage location. The user account includes a file-stage object that corresponds to the file stage and that includes an indication of the storage location. In response to receiving the request, the data platform creates the file-catalog table for the file stage. The file-catalog table lists the one or more files in the file stage, and includes at least two different metadata properties of the files listed in the file-catalog table.

公开(公告)号：US20210390095A1

公开(公告)日：2021-12-16

申请号：US17462435

申请日：2021-08-31

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F16/23 ,  G06F16/248 ,  G06F16/2458 ,  G06F3/06 ,  G06F16/25

Abstract: The subject technology receives a notification that a modification has been made to an external table, the modification comprising inserting at least one row of new data to the external table, the at least one row corresponding to a first micro-partition that includes a first portion of data from the external table prior to the inserting. The subject technology, in response to the notification indicating the modification to the external table, generates a new micro-partition different from the first micro-partition, the new micro-partition including the inserted at least one row of new data and the first portion of data from the external table. The subject technology generates a refreshed materialized view based at least in part on the generated new micro-partition such that the refreshed materialized view comprises a representation of the external table after the modification has been made.

公开(公告)号：US11163757B2

公开(公告)日：2021-11-02

申请号：US17200751

申请日：2021-03-12

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F3/06 ,  G06F16/2453 ,  G06F16/25 ,  G06F16/23 ,  G06F16/242

Abstract: Systems, methods, and devices for querying over an external table are disclosed. A method includes connecting a database platform to an external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes receiving a query comprising a predicate, the query directed at least to data in the external table. The method includes determining, based on metadata, one or more partitions in the external table comprising data satisfying the predicate. The method includes pruning, based on the metadata, all partitions in the external table that do not comprise any data satisfying the predicate. The method includes generating a query plan comprising a plurality of discrete subtasks. The method includes assigning, based on the metadata, the plurality of discrete subtasks to one or more nodes in an execution platform.

公开(公告)号：US20250068676A1

公开(公告)日：2025-02-27

申请号：US18933173

申请日：2024-10-31

Applicant: Snowflake Inc.

Inventor： Polita Paulus ,  Aravind Ramarathinam ,  Saurin Shah ,  Sukruth Komarla Sukumar

IPC: G06F16/901 ,  G06F16/22 ,  G06F16/2455 ,  G06F16/908 ,  G06F16/955

Abstract: Disclosed herein are embodiments of systems and methods for synchronizing file-catalog table with a file stage. In an embodiment, a data platform receives a notification of a modification to one or more files in a file stage. The file stage includes data storage having a storage location. The data platform updates, based on receiving the notification of the modification, a first file-catalog table for the file stage based on the modification. The first file-catalog table includes a row for each file in the file stage, as well as a column for each of one or more metadata properties of the one or more files in the file stage.

公开(公告)号：US12093281B2

公开(公告)日：2024-09-17

申请号：US18119775

申请日：2023-03-09

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Polita Paulus ,  Sahaj Saini ,  Saurin Shah ,  Srinidhi Karthik Bisthavalli Srinivasa

IPC: G06F16/20 ,  G06F16/25 ,  G06F16/27 ,  G06F16/955

CPC classification number: G06F16/27 ,  G06F16/254 ,  G06F16/9566

Abstract: Described herein are techniques for replicating external stages between deployments of e.g., a cloud data lake using a modified storage integration. The modified storage integration may be defined with a set of storage locations. The storage integration comprises a base URL for each of the set of storage locations and each storage location identifies a remote deployment where a cloud platform is hosted and a geographic region of the remote deployment. An external stage object may be bound to the storage integration, where the external stage object facilitates a data loading operation that is currently in progress on the first storage location. In response to detecting an outage at the first storage location, the data loading operation that was in progress on the first storage location may be continued at the second storage location using the storage integration.

公开(公告)号：US11960505B2

公开(公告)日：2024-04-16

申请号：US17664144

申请日：2022-05-19

Applicant: Snowflake Inc.

Inventor： Vasile Paraschiv ,  Saurin Shah ,  Marianne Shaw ,  Nileema Shingte

IPC: G06F16/27 ,  G06F9/30 ,  G06F16/13 ,  G06F16/182 ,  G06F16/22 ,  G06F16/2455 ,  G06F16/28 ,  G06F16/11 ,  G06F16/25

CPC classification number: G06F16/278 ,  G06F9/3009 ,  G06F16/137 ,  G06F16/182 ,  G06F16/2282 ,  G06F16/24554 ,  G06F16/283 ,  G06F16/116 ,  G06F16/254

Abstract: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.

公开(公告)号：US11675780B2

公开(公告)日：2023-06-13

申请号：US17650462

申请日：2022-02-09

Applicant: Snowflake Inc.

Inventor： Subramanian Muralidhar ,  Benoit Dageville ,  Thierry Cruanes ,  Nileema Shingte ,  Saurin Shah ,  Torsten Grabs ,  Istvan Cseri

IPC: G06F16/20 ,  G06F16/242 ,  G06F3/06 ,  G06F16/2453 ,  G06F16/25 ,  G06F16/23

CPC classification number: G06F16/2423 ,  G06F3/0605 ,  G06F3/067 ,  G06F3/0644 ,  G06F16/2393 ,  G06F16/24535 ,  G06F16/24542 ,  G06F16/254

Abstract: Disclosed herein are embodiments of systems and methods for partition-based scanning of external tables for query processing. In an example embodiment, a database platform receives a query that includes one or more predicates, where the query is directed at least to data in an external table that is stored in an external storage platform that is external to the database platform. The database platform identifies, based on metadata that summarizes the data in the external table, one or more partitions of the external table that potentially include data that satisfies the one or more predicates. The database platform also identifies, from the one or more identified partitions, data that satisfies the one or more predicates. The database platform sends a response to the query to the client, the response comprising the data satisfying the one or more predicates.

公开(公告)号：US11645413B2

公开(公告)日：2023-05-09

申请号：US17815031

申请日：2022-07-26

Applicant: Snowflake Inc.

Inventor： Abhiram Arikapudi ,  Saurin Shah

IPC: G06F21/62 ,  G06F21/31 ,  G06F21/44 ,  G06F21/46 ,  G06F21/78

CPC classification number: G06F21/6227 ,  G06F21/31 ,  G06F21/44 ,  G06F21/46 ,  G06F21/6245 ,  G06F21/78

Abstract: Techniques for document sharing include configuring a secure view of shared data as a listing in a data exchange of a data provider. The listing includes metadata with a first uniform resource locator (URL) of the shared data. An invitation to view the listing for is encoded transmission to a client device. A user defined function (UDF) client is invoked to access a data object of the shared data, based on detecting an acceptance of the invitation by the client device. A security function is applied to the data object to generate a modified data object. The metadata is updated to include a second URL, which is associated with a storage location storing the modified data object. The client device is granted access to the modified data object using the second URL, based on detecting a request by the client device to view the listing in the data exchange.