Invention Grant
- Patent Title: Export data from tables into partitioned folders on an external data lake
-
Application No.: US17086221Application Date: 2020-10-30
-
Publication No.: US11354331B2Publication Date: 2022-06-07
- Inventor: Vasile Paraschiv , Saurin Shah , Marianne Shaw , Nileema Shingte
- Applicant: Snowflake Inc.
- Applicant Address: US MT Bozeman
- Assignee: Snowflake Inc.
- Current Assignee: Snowflake Inc.
- Current Assignee Address: US MT Bozeman
- Agency: Schwegman Lundberg & Woessner, P.A.
- Main IPC: G06F16/25
- IPC: G06F16/25 ; G06F16/11 ; G06F16/182 ; G06F16/13

Abstract:
A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.
Public/Granted literature
- US20220121673A1 EXPORT DATA FROM TABLES INTO PARTITIONED FOLDERS ON AN EXTERNAL DATA LAKE Public/Granted day:2022-04-21
Information query