-
公开(公告)号:US12189614B2
公开(公告)日:2025-01-07
申请号:US18477834
申请日:2023-09-29
Applicant: Snowflake Inc.
Inventor: Khaled Yagoub , Wumengjian Zhu , Benoit Dageville , William Waddington
Abstract: The subject technology serializes, by at least one hardware processor, non-primary key data of column-organized data into compressed serialized value data that is in a row-organized sequence, the compressed serialized value data compressed using at least one bitmap, the non-primary key data comprising a schema identifier, the column-organized data being stored in a columnar database system, the column-organized data comprising primary key data and the non-primary key data. The subject technology stores the compressed serialized value data in a key-value data store of a key-value database system, the key-value database system processing key-value data in a key-value format. The subject technology receives a query by the columnar database system. The subject technology deserializes a portion of the compressed serialized value data that corresponds to the query. The subject technology processes the query using the columnar database system.
-
公开(公告)号:US20240028567A1
公开(公告)日:2024-01-25
申请号:US18326929
申请日:2023-05-31
Applicant: Snowflake Inc.
Inventor: Benoit Dageville , Adrian Hamza , Lishi Jiang , William Waddington , Khaled Yagoub , Wumengjian Zhu
CPC classification number: G06F16/213 , G06F16/221
Abstract: The subject technology generates, by a compute service manager, a schema hash value for a new schema version associated with a new schema version value, the schema hash value based on determining a sum of hash values of a set of attributes of value columns, the set of attributes comprises a column identifier, and a logical type of a column. The subject technology stores a mapping of the schema hash value to the new schema version value for a table in a metadata database. The subject technology stores a new schema entry based on the schema hash value, the new schema version value, and a new column for the table in the metadata database, the metadata database storing multiple entries for different schema versions, each entry including a particular schema hash value for mapping to a corresponding schema version from the different schema versions.
-
公开(公告)号:US11809414B2
公开(公告)日:2023-11-07
申请号:US17538818
申请日:2021-11-30
Applicant: Snowflake Inc.
Inventor: Khaled Yagoub , Wumengjian Zhu , Benoit Dageville , William Waddington
CPC classification number: G06F16/2379 , G06F11/1458 , G06F16/221 , G06F16/283
Abstract: A distributed database system can implement a column-based database system and a row-based database system for processing data. The row-based database system can store data organized into key value pairs, and data to be processed by the row-based database system is converted to a key-value format compressing keys that correspond to values. The distributed database system can perform serialization and compression in converting the data to the key-value format for efficient data storage performance. The distributed database system can unpack portions of the converted serialized compressed data in response to queries that process a portion of serialized compressed data without unpacking the entire converted dataset.
-
公开(公告)号:US11709808B1
公开(公告)日:2023-07-25
申请号:US17656558
申请日:2022-03-25
Applicant: Snowflake Inc.
Inventor: Benoit Dageville , Adrian Hamza , William Waddington , Khaled Yagoub , Wumengjian Zhu , Lishi Jiang
CPC classification number: G06F16/213 , G06F16/221
Abstract: The subject technology receives a statement to perform an operation to add a new column into a table. The subject technology generates a schema hash value for a new schema version associated with a new schema version value. The subject technology stores a mapping of the schema hash value to the new schema version value for the table in a metadata database. The subject technology stores a new schema entry based on the schema hash value, the new schema version value, and the new column for the table in the metadata database. The subject technology performs an operation to add the new column to the table.
-
公开(公告)号:US11675806B2
公开(公告)日:2023-06-13
申请号:US17249598
申请日:2021-03-05
Applicant: Snowflake Inc.
Inventor: Leonidas Galanis , Alexander Miller , William Waddington , Khaled Yagoub
IPC: G06F16/30 , G06F16/25 , G06F16/2452 , G06F16/28 , G06F16/2455 , G06F16/27
CPC classification number: G06F16/258 , G06F16/24524 , G06F16/24564 , G06F16/256 , G06F16/27 , G06F16/283
Abstract: A hybrid network-based database system for handling OLTP and OLAP queries using decoupled compute and storage devices. A set of decoupled compute instances perform transactions on an OLTP database, and the data is replicated to an OLAP database, which is managed by another set of decoupled compute instances. Further, in response to queries, the database system can retrieve data from the OLTP and OLAP database for merging and processing according to the query.
-
公开(公告)号:US20230169068A1
公开(公告)日:2023-06-01
申请号:US17538818
申请日:2021-11-30
Applicant: Snowflake Inc.
Inventor: Khaled Yagoub , Wumengjian Zhu , Benoit Dageville , William Waddington
CPC classification number: G06F16/2379 , G06F16/221 , G06F11/1458 , G06F16/283
Abstract: A distributed database system can implement a column-based database system and a row-based database system for processing data. The row-based database system can store data organized into key value pairs, and data to be processed by the row-based database system is converted to a key-value format compressing keys that correspond to values. The distributed database system can perform serialization and compression in converting the data to the key-value format for efficient data storage performance. The distributed database system can unpack portions of the converted serialized compressed data in response to queries that process a portion of serialized compressed data without unpacking the entire converted dataset.
-
公开(公告)号:US12135697B2
公开(公告)日:2024-11-05
申请号:US18326929
申请日:2023-05-31
Applicant: Snowflake Inc.
Inventor: Benoit Dageville , Adrian Hamza , Lishi Jiang , William Waddington , Khaled Yagoub , Wumengjian Zhu
Abstract: The subject technology generates, by a compute service manager, a schema hash value for a new schema version associated with a new schema version value, the schema hash value based on determining a sum of hash values of a set of attributes of value columns, the set of attributes comprises a column identifier, and a logical type of a column. The subject technology stores a mapping of the schema hash value to the new schema version value for a table in a metadata database. The subject technology stores a new schema entry based on the schema hash value, the new schema version value, and a new column for the table in the metadata database, the metadata database storing multiple entries for different schema versions, each entry including a particular schema hash value for mapping to a corresponding schema version from the different schema versions.
-
公开(公告)号:US20240020298A1
公开(公告)日:2024-01-18
申请号:US18477834
申请日:2023-09-29
Applicant: Snowflake Inc.
Inventor: Khaled Yagoub , Wumengjian Zhu , Benoit Dageville , William Waddington
CPC classification number: G06F16/2379 , G06F16/283 , G06F11/1458 , G06F16/221
Abstract: The subject technology serializes, by at least one hardware processor, non-primary key data of column-organized data into compressed serialized value data that is in a row-organized sequence, the compressed serialized value data compressed using at least one bitmap, the non-primary key data comprising a schema identifier, the column-organized data being stored in a columnar database system, the column-organized data comprising primary key data and the non-primary key data. The subject technology stores the compressed serialized value data in a key-value data store of a key-value database system, the key-value database system processing key-value data in a key-value format. The subject technology receives a query by the columnar database system. The subject technology deserializes a portion of the compressed serialized value data that corresponds to the query. The subject technology processes the query using the columnar database system.
-
公开(公告)号:US20230267130A1
公开(公告)日:2023-08-24
申请号:US18309056
申请日:2023-04-28
Applicant: Snowflake Inc.
Inventor: Leonidas Galanis , Alexander Miller , William Waddington , Khaled Yagoub
IPC: G06F16/25 , G06F16/2452 , G06F16/28 , G06F16/2455 , G06F16/27
CPC classification number: G06F16/258 , G06F16/256 , G06F16/24524 , G06F16/283 , G06F16/24564 , G06F16/27
Abstract: The subject technology receives, by a first set of decoupled compute instances of a distributed database, a plurality of transactional queries against a transactional processing database stored on a first set of decoupled storage instances, at least one of the first set of decoupled compute instances bypassing a compiler of the distributed database to process the plurality of transactional queries using locally cached data. The subject technology receives at least one select query for processing on an aggregated analytical query database of the distributed database. The subject technology generates, by the compiler of the distributed database, a query plan for execution of the at least one select query using a second set of decoupled compute instances. The subject technology generates, by the second set of decoupled compute instances, an aggregated analytical query result dataset for the at least one select query according to the query plan.
-
公开(公告)号:US20220188325A1
公开(公告)日:2022-06-16
申请号:US17249598
申请日:2021-03-05
Applicant: Snowflake Inc.
Inventor: Leonidas Galanis , Alexander Miller , William Waddington , Khaled Yagoub
IPC: G06F16/25 , G06F16/27 , G06F16/28 , G06F16/2455 , G06F16/2452
Abstract: A hybrid network-based database system for handling OLTP and OLAP queries using decoupled compute and storage devices. A set of decoupled compute instances perform transactions on an OLTP database, and the data is replicated to an OLAP database, which is managed by another set of decoupled compute instances. Further, in response to queries, the database system can retrieve data from the OLTP and OLAP database for merging and processing according to the query.
-
-
-
-
-
-
-
-
-