公开(公告)号：US20220179846A1

公开(公告)日：2022-06-09

申请号：US17652504

申请日：2022-02-25

Applicant: Snowflake Inc.

Inventor： Jiansheng Huang ,  Jiaxing Liang ,  Scott Ziegler ,  Haowei Yu ,  Benoit Dageville ,  Varun Ganesh

IPC: G06F16/23 ,  G06F16/22 ,  G06F16/25

Abstract: Systems, methods, and devices for batch ingestion of data into a table of a database. A method includes determining a notification indicating a presence of a user file received from a client account to be ingested into a database. The method includes identifying data in the user file and identifying a target table of the database to receive the data in the user file. The method includes generating an ingest task indicating the data and the target table. The method includes assigning the ingest task to an execution node of an execution platform, wherein the execution platform comprises a plurality of execution nodes operating independent of a plurality of shared storage devices collectively storing database data. The method includes registering metadata concerning the target table in a metadata store after the data has been fully committed to the target table by the execution node.

公开(公告)号：US20200242118A1

公开(公告)日：2020-07-30

申请号：US16851800

申请日：2020-04-17

Applicant: SNOWFLAKE INC.

Inventor： Benoit Dageville ,  Jiansheng Huang ,  Shige Takeda ,  Abdul Q. Munir

IPC: G06F16/2453 ,  G06F16/11 ,  G06F16/16 ,  G06F16/2455 ,  G06F16/14

Abstract: A method, system, and apparatus for managing a persistent query result set are disclosed. A resource manager is disclosed that receives a query from a client over a computer network. A computer database server is disclosed that executes the query and generates a result set based on and in response to the received query. An execution platform is disclosed that divides the result set into a plurality of chunks, wherein each chunk comprises a portion of the generated result set. The resource manager adds metadata to each of the plurality of chunks. A first chunk of the plurality of chunks comprises metadata pointing to the plurality of chunks. The resource manager delivers at least the first chunk of the plurality of chunks to the client in response to the query. The execution platform offloads the remaining plurality of chunks to cloud storage in parallel.

公开(公告)号：US11055280B2

公开(公告)日：2021-07-06

申请号：US16201854

申请日：2018-11-27

Applicant: Snowflake Inc.

Inventor： Jiansheng Huang ,  Jiaxing Liang ,  Scott Ziegler ,  Haowei Yu ,  Benoit Dageville ,  Varun Ganesh

IPC: G06F16/23 ,  G06F16/22 ,  G06F16/25

Abstract: Systems, methods, and devices for batch ingestion of data into a table of a database. A method includes determining a notification indicating a presence of a user file received from a client account to be ingested into a database. The method includes identifying data in the user file and identifying a target table of the database to receive the data in the user file. The method includes generating an ingest task indicating the data and the target table. The method includes assigning the ingest task to an execution node of an execution platform, wherein the execution platform comprises a plurality of execution nodes operating independent of a plurality of shared storage devices collectively storing database data. The method includes registering metadata concerning the target table in a metadata store after the data has been fully committed to the target table by the execution node.

公开(公告)号：US10896172B2

公开(公告)日：2021-01-19

申请号：US16720418

申请日：2019-12-19

Applicant: Snowflake Inc.

Inventor： Benoit Dageville ,  Varun Ganesh ,  Jiansheng Huang ,  Jiaxing Liang ,  Haowei Yu ,  Scott Ziegler

IPC: G06F16/23 ,  G06F16/22 ,  G06F16/25

Abstract: Systems, methods, and devices for batch ingestion of data into a table of a database. A method includes determining a notification indicating a presence of a user file received from a client account to be ingested into a database. The method includes identifying data in the user file and identifying a target table of the database to receive the data in the user file. The method includes generating an ingest task indicating the data and the target table. The method includes assigning the ingest task to an execution node of an execution platform, wherein the execution platform comprises a plurality of execution nodes operating independent of a plurality of shared storage devices collectively storing database data. The method includes registering metadata concerning the target table in a metadata store after the data has been fully committed to the target table by the execution node.

公开(公告)号：US11836132B2

公开(公告)日：2023-12-05

申请号：US16851800

申请日：2020-04-17

Applicant: SNOWFLAKE INC.

Inventor： Benoit Dageville ,  Jiansheng Huang ,  Shige Takeda ,  Abdul Q. Munir

IPC: G06F16/2453 ,  G06F16/11 ,  G06F16/14 ,  G06F16/16 ,  G06F16/2455 ,  H04L67/06 ,  H04L67/1097

CPC classification number: G06F16/24539 ,  G06F16/125 ,  G06F16/156 ,  G06F16/162 ,  G06F16/2455

Abstract: A method, system, and apparatus for managing a persistent query result set are disclosed. A resource manager is disclosed that receives a query from a client over a computer network. A computer database server is disclosed that executes the query and generates a result set based on and in response to the received query. An execution platform is disclosed that divides the result set into a plurality of chunks, wherein each chunk comprises a portion of the generated result set. The resource manager adds metadata to each of the plurality of chunks. A first chunk of the plurality of chunks comprises metadata pointing to the plurality of chunks. The resource manager delivers at least the first chunk of the plurality of chunks to the client in response to the query. The execution platform offloads the remaining plurality of chunks to cloud storage in parallel.

公开(公告)号：US11294890B2

公开(公告)日：2022-04-05

申请号：US16365219

申请日：2019-03-26

Applicant: Snowflake Inc.

Inventor： Jiansheng Huang ,  Jiaxing Liang ,  Scott Ziegler ,  Haowei Yu ,  Benoit Dageville ,  Varun Ganesh

IPC: G06F16/23 ,  G06F16/22 ,  G06F16/25

Abstract: Systems, methods, and devices for batch ingestion of data into a table of a database. A method includes determining a notification indicating a presence of a user file received from a client account to be ingested into a database. The method includes identifying data in the user file and identifying a target table of the database to receive the data in the user file. The method includes generating an ingest task indicating the data and the target table. The method includes assigning the ingest task to an execution node of an execution platform, wherein the execution platform comprises a plurality of execution nodes operating independent of a plurality of shared storage devices collectively storing database data. The method includes registering metadata concerning the target table in a metadata store after the data has been fully committed to the target table by the execution node.

公开(公告)号：US10997163B2

公开(公告)日：2021-05-04

申请号：US16943251

申请日：2020-07-30

Applicant: Snowflake Inc.

Inventor： Benoit Dageville ,  Varun Ganesh ,  Jiansheng Huang ,  Jiaxing Liang ,  Haowei Yu ,  Scott Ziegler

IPC: G06F17/00 ,  G06F16/23 ,  G06F16/22 ,  G06F16/25

Abstract: The subject technology at a data system, an ingest request to ingest one or more files into a table. The subject technology, after obtaining the ingest request and prior to the ingesting of the one or more files, persists the one or more files in a first file queue that corresponds to the table, the first file queue further corresponding to a client account, and the data system further comprising a second file queue that corresponds to both a second client account and a second table. The subject technology ingests, by one or more execution nodes, the one or more files into one or more micro-partitions of the table, each of the one or more micro-partitions comprising contiguous units of storage of a storage device.

公开(公告)号：US10977245B2

公开(公告)日：2021-04-13

申请号：US16942421

申请日：2020-07-29

Applicant: Snowflake Inc.

Inventor： Benoit Dageville ,  Varun Ganesh ,  Jiansheng Huang ,  Jiaxing Liang ,  Haowei Yu ,  Scott Ziegler

IPC: G06F17/00 ,  G06F16/23 ,  G06F16/22 ,  G06F16/25

Abstract: The subject technology obtains, at a database system, an ingest request to ingest one or more files into a table of a database. The subject technology, after obtaining the ingest request and prior to the ingesting of the one or more files, persists the one or more files in a file queue that corresponds to the table. The subject technology assigns the one or more files to one or more execution nodes to be ingested into the table. The subject technology operates an ingest puller to poll the file queue. The subject technology ingests, by the one or more execution nodes, the one or more files into one or more micro-partitions of the table via one or more pipes.

公开(公告)号：US10671605B2

公开(公告)日：2020-06-02

申请号：US15097107

申请日：2016-04-12

Applicant: SNOWFLAKE INC.

Inventor： Benoit Dageville ,  Jiansheng Huang ,  Shige Takeda ,  Abdul Q. Munir

IPC: G06F16/2453 ,  G06F16/14 ,  G06F16/2455 ,  G06F16/11 ,  G06F16/16

Abstract: A method, system, and apparatus for managing a persistent query result set are disclosed. A resource manager is disclosed that receives a query from a client over a computer network. A computer database server is disclosed that executes the query and generates a result set based on and in response to the received query. An execution platform is disclosed that divides the result set into a plurality of chunks, wherein each chunk comprises a portion of the generated result set. The resource manager adds metadata to each of the plurality of chunks. A first chunk of the plurality of chunks comprises metadata pointing to the plurality of chunks. The resource manager delivers at least the first chunk of the plurality of chunks to the client in response to the query. The execution platform offloads the remaining plurality of chunks to cloud storage in parallel.