-
公开(公告)号:US11347775B2
公开(公告)日:2022-05-31
申请号:US17463313
申请日:2021-08-31
Applicant: Snowflake Inc.
Inventor: Vasile Paraschiv , Saurin Shah , Marianne Shaw , Nileema Shingte
IPC: G06F16/27 , G06F16/28 , G06F9/30 , G06F16/22 , G06F16/2455
Abstract: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.
-
公开(公告)号:US20220150252A1
公开(公告)日:2022-05-12
申请号:US17587829
申请日:2022-01-28
Applicant: Snowflake Inc.
Inventor: Polita Paulus , Peter Povinec , Saurin Shah , Srinidhi Karthik Bisthavalli Srinivasa
Abstract: A command to load or unload data at a storage location is received. In response to the command, a storage integration object associated with the storage location is identified. The storage integration object identifies a cloud identity object that corresponds to a cloud identity that is associated with a proxy identity object corresponding to a proxy identity granted permission to access the storage location. The data is loaded or unloaded at the storage location by assuming the proxy identity.
-
公开(公告)号:US11269869B2
公开(公告)日:2022-03-08
申请号:US17498382
申请日:2021-10-11
Applicant: Snowflake Inc.
Inventor: Subramanian Muralidhar , Benoit Dageville , Thierry Cruanes , Nileema Shingte , Saurin Shah , Torsten Grabs , Istvan Cseri
IPC: G06F16/20 , G06F16/242 , G06F3/06 , G06F16/2453 , G06F16/25 , G06F16/23
Abstract: Disclosed herein are systems and methods for processing queries over external tables. In an embodiment, a database platform receives a query directed at least to data in an external table stored in a storage platform that is external to the database platform. The database platform uses metadata that summarizes the data in the external table to identify one or more partitions of the external table as potentially including data satisfying the query, and generates a query plan that includes a plurality of discrete subtasks that collectively include instructions to scan the identified one or more partitions of the external table for data satisfying the query. The database platform assigns, based on the metadata, the plurality of discrete subtasks to one or more nodes in an execution platform, and refreshes the metadata in response to a threshold number of modifications being made to the external table.
-
公开(公告)号:US20210211425A1
公开(公告)日:2021-07-08
申请号:US17249971
申请日:2021-03-19
Applicant: Snowflake Inc.
Inventor: Polita Paulus , Peter Povinec , Saurin Shah , Srinidhi Karthik Bisthavalli Srinivasa
Abstract: A command to load or unload data at a storage location is received. In response to the command, a storage integration object associated with the storage location is identified. The storage integration object identifies a cloud identity object that corresponds to a cloud identity that is associated with a proxy identity object corresponding to a proxy identity granted permission to access the storage location. The data is loaded or unloaded at the storage location by assuming the proxy identity.
-
公开(公告)号:US20210200754A1
公开(公告)日:2021-07-01
申请号:US17200751
申请日:2021-03-12
Applicant: Snowflake Inc.
Inventor: Subramanian Muralidhar , Benoit Dageville , Thierry Cruanes , Nileema Shingte , Saurin Shah , Torsten Grabs , Istvan Cseri
IPC: G06F16/242 , G06F3/06 , G06F16/2453 , G06F16/25 , G06F16/23
Abstract: Systems, methods, and devices for querying over an external table are disclosed. A method includes connecting a database platform to an external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes receiving a query comprising a predicate, the query directed at least to data in the external table. The method includes determining, based on metadata, one or more partitions in the external table comprising data satisfying the predicate. The method includes pruning, based on the metadata, all partitions in the external table that do not comprise any data satisfying the predicate. The method includes generating a query plan comprising a plurality of discrete subtasks. The method includes assigning, based on the metadata, the plurality of discrete subtasks to one or more nodes in an execution platform.
-
公开(公告)号:US10997243B1
公开(公告)日:2021-05-04
申请号:US16942352
申请日:2020-07-29
Applicant: Snowflake Inc.
Inventor: Polita Paulus , Aravind Ramarathinam , Saurin Shah , Sukruth Komarla Sukumar
IPC: G06F16/00 , G06F16/901 , G06F16/955 , G06F16/2455 , G06F16/22 , G06F16/908
Abstract: A data platform provides support for unstructured files, semi-structured files, and structured files. In various different embodiments, the data platform provides functions for obtaining relative paths, absolute paths, and presigned URLs for files in internal and external stages; application programming interfaces (APIs) for integrating with external functions; built-in file-catalog tables; integration with cloud-storage platforms; customized views for flexible file-sharing options, stage-encryption options; notification integrations; table functions for generating file-catalog tables; streams on stages; streams on tables; and other features.
-
公开(公告)号:US20200334231A1
公开(公告)日:2020-10-22
申请号:US16775110
申请日:2020-01-28
Applicant: Snowflake Inc.
Inventor: Subramanian Muralidhar , Benoit Dageville , Thierry Cruanes , Nileema Shingte , Saurin Shah , Torsten Grabs , Istvan Cseri
IPC: G06F16/23 , G06F16/248 , G06F16/2458 , G06F16/25 , G06F3/06
Abstract: Systems, methods, and devices for generating a materialized view over an external table. A method includes connecting a database platform to an external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes generating, by the database platform, a materialized view over the external table. The method includes receiving a notification that a modification has been made to the external table, the modification comprising one or more of an addition of a file, a deletion of a file, or an update to a file in a source location for the external table. The method includes, in response to the external table being modified, refreshing the materialized view such that the materialized view comprises an accurate representation of the external table.
-
公开(公告)号:US20200334230A1
公开(公告)日:2020-10-22
申请号:US16385720
申请日:2019-04-16
Applicant: Snowflake Inc.
Inventor: Subramanian Muralidhar , Benoit Dageville , Thierry Cruanes , Nileema Shingte , Saurin Shah , Torsten Grabs , Istvan Cseri
IPC: G06F16/23 , G06F16/25 , G06F16/248 , G06F16/2458 , G06F3/06
Abstract: Systems, methods, and devices for generating a materialized view over an external table. A method includes connecting a database platform to an external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes generating, by the database platform, a materialized view over the external table. The method includes receiving a notification that a modification has been made to the external table, the modification comprising one or more of an addition of a file, a deletion of a file, or an update to a file in a source location for the external table. The method includes, in response to the external table being modified, refreshing the materialized view such that the materialized view comprises an accurate representation of the external table.
-
公开(公告)号:US12236355B2
公开(公告)日:2025-02-25
申请号:US18416379
申请日:2024-01-18
Applicant: Snowflake Inc.
Inventor: Michal Gdak , Ganeshan Ramachandran Iyer , Tomasz Malisz , Mikolaj Niedbala , Pawel Pollak , Saurin Shah , Jan Tomasz Topinski , Daria Wieteska
Abstract: Systems and methods for generating a machine-learning (ML) model for extracting information from one or more electronic documents, where the ML model can be used as a data object, which can be part of a database command or as part of a document information extraction process that is continuously running (e.g., document information extraction pipeline).
-
公开(公告)号:US20250061226A1
公开(公告)日:2025-02-20
申请号:US18935906
申请日:2024-11-04
Applicant: Snowflake Inc.
Inventor: Abhiram Arikapudi , Saurin Shah
Abstract: Techniques for secure sharing of a data exchange listing includes retrieving a listing policy for a data listing of a private data exchange associated with a data provider. Metadata of the data listing is generated. The metadata includes an access control configuration for the data listing. The access control configuration is based at least on the listing policy. A security function is invoked to stage data of the data listing based on an access request for the data listing. The invoking causes a modification of the stage data to obtain modified stage data. The access control configuration is updated based on the modification.
-
-
-
-
-
-
-
-
-