公开(公告)号：US10810232B2

公开(公告)日：2020-10-20

申请号：US16775092

申请日：2020-01-28

Applicant: Snowflake Inc.

Inventor： Ashish Motivala ,  Yi Fang ,  Martin Hentschel ,  Benoit Dageville ,  Spyros Triantafylis ,  Yizhi Zhu

IPC: G06F16/27 ,  G06F16/245 ,  G06F16/22

Abstract: Systems, methods, and devices for multiple level metadata organization in database systems. A system includes a plurality of shared storage devices collectively storing database data across a plurality of tables that each comprise one or more micro-partitions including immutable storage devices. The system includes a resource manager configured to manage the storage of database data stored in a table across one or more of the plurality of shared storage devices. The resource manager is configured to manage the storage of metadata for the table. The metadata includes a column expression property with information about data stored in a column, a micro-partition expression property with information about data stored in a micro-partition, a grouping expression property with information about data stored in two or more micro-partitions, and cumulative table metadata with global information about all micro-partitions of the table.

公开(公告)号：US20240354315A1

公开(公告)日：2024-10-24

申请号：US18302234

申请日：2023-04-18

Applicant: SNOWFLAKE INC.

Inventor： Varun Ganesh ,  Alvin E. Jou ,  Donghe Kang ,  Ryan Michael Thomas Shelly ,  Jiaqi Yan ,  Yizhi Zhu

IPC: G06F16/28 ,  G06F16/2455

CPC classification number: G06F16/285 ,  G06F16/24556

Abstract: A method for selecting micro-partitions for a clustering operation includes: storing table data in a plurality of micro-partitions of a storage device, wherein each of the plurality of micro-partitions comprises a portion of the table data, wherein subsets of the plurality of micro-partitions are associated with a respective one of a plurality of expression property (EP) files, and wherein each of the plurality of EP files comprises an EP data region that represents the portions of the table data of the subset of the plurality of micro-partitions associated with the EP file; determining sub-ranges of the table data based on the EP data regions of the plurality of EP files; selecting a subset of the plurality of EP files for a clustering operation based on the sub-ranges of the table data; and performing the clustering operation on the micro-partitions associated with the subset of the EP files.

公开(公告)号：US11636114B2

公开(公告)日：2023-04-25

申请号：US17123551

申请日：2020-12-16

Applicant: Snowflake Inc.

Inventor： Benoit Dageville ,  Yi Fang ,  Martin Hentschel ,  Ashish Motivala ,  Spyridon Triantafyllis ,  Yizhi Zhu

IPC: G06F16/2455 ,  G06F16/23 ,  G06F16/2457 ,  G06F16/22 ,  G06F16/2458 ,  G06F16/27

Abstract: The subject technology receives first metadata corresponding to a set of micro-partitions. The subject technology stores a first data structure and a second data structure in storage as a first file and a second file, first data structure including the first metadata and a second data structure including second metadata, the first metadata corresponding to a set of micro-partitions, the second metadata for a grouping of the first metadata, the second data structure including information associating the second metadata to the first metadata. The subject technology stores third metadata for a table, the third metadata comprising information about data stored in a micro-partition of the table.

公开(公告)号：US11106678B2

公开(公告)日：2021-08-31

申请号：US17086279

申请日：2020-10-30

Applicant: Snowflake Inc.

Inventor： Benoit Dageville ,  Yi Fang ,  Martin Hentschel ,  Ashish Motivala ,  Spyridon Triantafyllis ,  Yizhi Zhu

IPC: G06F16/2455 ,  G06F16/23 ,  G06F16/2457 ,  G06F16/22 ,  G06F16/2458 ,  G06F16/27

Abstract: The subject technology receives first metadata corresponding to a set of micro-partitions. The subject technology stores a first data structure and a second data structure in storage as a first file and a second file, first data structure including the first metadata and a second data structure including second metadata, the first metadata corresponding to a set of micro-partitions, the second metadata for a grouping of the first metadata, the second data structure including information associating the second metadata to the first metadata. The subject technology stores third metadata for a table, the third metadata comprising: cumulative table metadata comprising global information about a plurality of micro-partitions of the table, the cumulative table metadata being stored in a metadata micro-partition associated with the table.

公开(公告)号：US20210103589A1

公开(公告)日：2021-04-08

申请号：US17123551

申请日：2020-12-16

Applicant: Snowflake Inc.

Inventor： Benoit Dageville ,  Yi Fang ,  Martin Hentschel ,  Ashish Motivala ,  Spyros Triantafyllis ,  Yizhi Zhu

IPC: G06F16/2455 ,  G06F16/23 ,  G06F16/2457 ,  G06F16/22 ,  G06F16/2458 ,  G06F16/27

Abstract: The subject technology receives first metadata corresponding to a set of micro-partitions. The subject technology stores a first data structure and a second data structure in storage as a first file and a second file, first data structure including the first metadata and a second data structure including second metadata, the first metadata corresponding to a set of micro-partitions, the second metadata for a grouping of the first metadata, the second data structure including information associating the second metadata to the first metadata. The subject technology stores third metadata for a table, the third metadata comprising information about data stored in a micro-partition of the table.

公开(公告)号：US20210019334A1

公开(公告)日：2021-01-21

申请号：US17062859

申请日：2020-10-05

Applicant: Snowflake Inc.

Inventor： Ashish Motivala ,  Yi Fang ,  Martin Hentschel ,  Benoit Dageville ,  Spyros Triantafyllis ,  Yizhi Zhu

IPC: G06F16/27 ,  G06F16/245 ,  G06F16/22

Abstract: The subject technology stores database data in a table, the table comprising a plurality of micro-partitions each having rows and columns. The subject technology stores metadata for the table in a data structure, the metadata comprising a first level of the data structure comprising a column expression property comprising information about data stored in a column of a micro-partition of the table, a micro-partition expression property comprising information about data stored in the micro-partition of the table, and a second level of the data structure comprising a grouping expression property comprising information about data stored in the micro-partition and one or more additional micro-partitions of the table.

公开(公告)号：US20200272638A1

公开(公告)日：2020-08-27

申请号：US16775092

申请日：2020-01-28

Applicant: Snowflake Inc

Inventor： Ashish Motivala ,  Yi Fang ,  Martin Hentschel ,  Benoit Dageville ,  Spyros Triantafylis ,  Yizhi Zhu

IPC: G06F16/27 ,  G06F16/22 ,  G06F16/245

Abstract: Systems, methods, and devices for multiple level metadata organization in database systems. A system includes a plurality of shared storage devices collectively storing database data across a plurality of tables that each comprise one or more micro-partitions including immutable storage devices. The system includes a resource manager configured to manage the storage of database data stored in a table across one or more of the plurality of shared storage devices. The resource manager is configured to manage the storage of metadata for the table. The metadata includes a column expression property with information about data stored in a column, a micro-partition expression property with information about data stored in a micro-partition, a grouping expression property with information about data stored in two or more micro-partitions, and cumulative table metadata with global information about all micro-partitions of the table.

公开(公告)号：US20200272628A1

公开(公告)日：2020-08-27

申请号：US16863831

申请日：2020-04-30

Applicant: Snowflake Inc.

Inventor： Benoit Dageville ,  Yi Fang ,  Martin Hentschel ,  Ashish Motivala ,  Spyridon Triantafyllis ,  Yizhi Zhu

IPC: G06F16/2455 ,  G06F16/23

Abstract: The subject technology receives first metadata corresponding to a set of micro-partitions. The subject technology generates second metadata for a grouping of the first metadata. The subject technology generates a first data structure including the first metadata and a second data structure including the second metadata, the second data structure including information associating the second metadata to the first metadata. The subject technology stores the first data structure and the second data structure in persistent storage as a first file and a second file. The subject technology receives a query on a table. Further, the subject technology analyzes the query against cumulative table metadata to determine whether data stored in the table matches the query.