Massively Parallel And In-Memory Execution Of Grouping And Aggregation In a Heterogeneous System

    公开(公告)号:US20190188205A1

    公开(公告)日:2019-06-20

    申请号:US16272829

    申请日:2019-02-11

    CPC classification number: G06F16/24556 G06F16/24553

    Abstract: A system and method for processing a group and aggregate query on a relation are disclosed. A database system determines whether assistance of a heterogeneous system (HS) of compute nodes is beneficial in performing the query. Assuming that the relation has been partitioned and loaded into the HS, the database system determines, in a compile phase, whether the HS has the functional capabilities to assist, and whether the cost and benefit favor performing the operation with the assistance of the HS. If the cost and benefit favor using the assistance of the HS, then the system enters the execution phase. The database system starts, in the execution phase, an optimal number of parallel processes to produce and consume the results from the compute nodes of the HS. After any needed transaction consistency checks, the results of the query are returned by the database system.

    Data recovery for a relational database management system instance in a heterogeneous database system
    13.
    发明授权
    Data recovery for a relational database management system instance in a heterogeneous database system 有权
    异构数据库系统中关系数据库管理系统实例的数据恢复

    公开(公告)号:US09563522B2

    公开(公告)日:2017-02-07

    申请号:US14675490

    申请日:2015-03-31

    Abstract: A method and apparatus for data recovery for a RDBMS instance in a heterogeneous database system is provided. A failure of a first RDBMS instance is detected in a plurality of RDBMS instances of a shared-disk database system. A compute cluster is configured to store, in memory, one or more tables stored by the shared-disk database system. The first RDBMS instance is configured to modify the one or more tables stored by the shared-disk database system and transfer modified data to the compute cluster to update the one or more tables at the compute cluster. After detecting the failure of the first RDBMS instance, redo records generated by the first RDBMS instance are scanned, pending modified data that was not transferred to the compute cluster before the failure is identified, and the pending modified data is transferred to the compute cluster.

    Abstract translation: 提供了用于异构数据库系统中的RDBMS实例的数据恢复的方法和装置。 在共享磁盘数据库系统的多个RDBMS实例中检测到第一RDBMS实例的故障。 计算集群配置为在内存中存储由共享磁盘数据库系统存储的一个或多个表。 第一个RDBMS实例被配置为修改由共享磁盘数据库系统存储的一个或多个表,并将修改的数据传输到计算集群,以更新计算集群中的一个或多个表。 在检测到第一个RDBMS实例的故障之后,将扫描由第一个RDBMS实例生成的重做记录,在发现故障之前尚未传输到计算集群的待修改数据,并将待处理的修改数据传输到计算集群。

    Massively Parallel And In-Memory Execution Of Grouping And Aggregation In a Heterogeneous System
    14.
    发明申请
    Massively Parallel And In-Memory Execution Of Grouping And Aggregation In a Heterogeneous System 审中-公开
    在非均匀系统中进行大规模并行和内存中的分组和聚合执行

    公开(公告)号:US20140280298A1

    公开(公告)日:2014-09-18

    申请号:US13831122

    申请日:2013-03-14

    CPC classification number: G06F17/30489 G06F17/30483

    Abstract: A system and method for processing a group and aggregate query on a relation are disclosed. A database system determines whether assistance of a heterogeneous system (HS) of compute nodes is beneficial in performing the query. Assuming that the relation has been partitioned and loaded into the HS, the database system determines, in a compile phase, whether the HS has the functional capabilities to assist, and whether the cost and benefit favor performing the operation with the assistance of the HS. If the cost and benefit favor using the assistance of the HS, then the system enters the execution phase. The database system starts, in the execution phase, an optimal number of parallel processes to produce and consume the results from the compute nodes of the HS. After any needed transaction consistency checks, the results of the query are returned by the database system.

    Abstract translation: 公开了一种用于处理关系的组和聚合查询的系统和方法。 数据库系统确定计算节点的异构系统(HS)的协助是否有利于执行查询。 假设该关系已经被划分并加载到HS中,数据库系统在编译阶段确定HS是否具有协助的功能能力,以及成本和利益是否有利于在HS的协助下执行操作。 如果成本和收益有利于使用HS的协助,则系统进入执行阶段。 数据库系统在执行阶段启动最佳数量的并行进程,以产生和消耗来自HS的计算节点的结果。 在任何所需的事务一致性检查之后,查询的结果由数据库系统返回。

    Efficient File Access In A Large Repository Using A Two-Level Cache
    15.
    发明申请
    Efficient File Access In A Large Repository Using A Two-Level Cache 审中-公开
    使用两级缓存的大型存储库中的高效文件访问

    公开(公告)号:US20130097175A1

    公开(公告)日:2013-04-18

    申请号:US13692014

    申请日:2012-12-03

    CPC classification number: G06F17/30097 G06F12/0811 G06F12/084 G06F17/30929

    Abstract: A two-level cache to facilitate resolving resource path expressions for a hierarchy of resources is described, which includes a system-wide shared cache and a session-level cache. The shared cache is organized as a hierarchy of hash tables that mirrors the structure of a repository hierarchy. A particular hash table in a shared cache includes information for the child resources of a particular resource. A database management system that manages a shared cache may control the amount of memory used by the cache by implementing a replacement policy for the cache based on one or more characteristics of the resources in the repository. The session-level cache is a single level cache in which information for target resources of resolved path expressions may be tracked. In the session-level cache, the resource information is associated with the entire path expression of the associated resource.

    Abstract translation: 描述了用于促进解决资源层级的资源路径表达式的两级缓存,其包括系统范围共享高速缓存和会话级缓存。 共享缓存被组织为映射存储库层次结构的散列表的层次结构。 共享缓存中的特定哈希表包括特定资源的子资源的信息。 管理共享高速缓存的数据库管理系统可以基于存储库中的资源的一个或多个特性来实现对高速缓存的替换策略来控制高速缓存所使用的存储器量。 会话级缓存是单级缓存,其中可以跟踪解析的路径表达式的目标资源的信息。 在会话级缓存中,资源信息与相关资源的整个路径表达式相关联。

    Sparse dictionary tree
    16.
    发明授权

    公开(公告)号:US11023430B2

    公开(公告)日:2021-06-01

    申请号:US15819891

    申请日:2017-11-21

    Abstract: Techniques related to a sparse dictionary tree are disclosed. In some embodiments, computing device(s) execute instructions, which are stored on non-transitory storage media, for performing a method. The method comprises storing an encoding dictionary as a token-ordered tree comprising a first node and a second node, which are adjacent nodes. The token-ordered tree maps ordered tokens to ordered codes. The ordered tokens include a first token and a second token. The ordered codes include a first code and a second code, which are non-consecutive codes. The first node maps the first token to the first code. The second node maps the second token to the second code. The encoding dictionary is updated based on inserting a third node between the first node and the second node. The third node maps a third token to a third code that is greater than the first code and less than the second code.

    BITMAP-BASED COUNT DISTINCT QUERY REWRITE IN A RELATIONAL SQL ALGEBRA

    公开(公告)号:US20210109930A1

    公开(公告)日:2021-04-15

    申请号:US16653639

    申请日:2019-10-15

    Abstract: Techniques are described for storing and maintaining, in a materialized view, bitmap data that represents a bitmap of each possible distinct value of an expression and rewriting a query for a count of distinct values of the expression using the materialized view. The materialized view contains bitmap data that represents a bitmap of each possible distinct value of a first expression, and aggregate values of additional expressions, and is stored in memory or on disk by a database system. The database system receives a query that requests a number of distinct values, of the first expression, and an aggregate value for an additional expression. In response, the database system, rewrites the query to: compute the number of distinct values by counting the bits in the bitmap data of the materialized view that are set to the first value, and obtains the aggregate value for the additional expression in the materialized view.

    Workload aware data placement for join-based query processing in a cluster

    公开(公告)号:US09984081B2

    公开(公告)日:2018-05-29

    申请号:US14610892

    申请日:2015-01-30

    Abstract: A method for distributing tables to a cluster of nodes managed by database management system (DBMS), is disclosed. Multiple data placement schemes are evaluated based on a query workload set to select a data placement scheme for the cluster of nodes. Tables, used in join operations in the workload set, are selected for evaluation of data placement schemes. Query execution costs for the workload set are generated based on estimating a query execution cost for each data placement scheme for the tables. The data placement scheme that has least costly estimated execution cost for the workload set is selected as the data placement scheme for the cluster of nodes managed by DBMS.

    Method for failure-resilient data placement in a distributed query processing system

    公开(公告)号:US09842148B2

    公开(公告)日:2017-12-12

    申请号:US14704825

    申请日:2015-05-05

    CPC classification number: G06F17/30545

    Abstract: Herein is described a data placement scheme for a distributed query processing systems that achieves load balance amongst the nodes of the system. To identify a node on which to place particular data, a supervisor node performs a placement algorithm over the particular data's identifier, where the placement algorithm utilizes two or more hash functions. The supervisor node runs the placement algorithm until a destination node is identified that is available to store the data, or the supervisor node has run the placement algorithm an established number of times. If no available node is identified using the placement algorithm, then an available destination node is identified for the particular data and information identifying the data and the selected destination node is included in an exception map. Most data may be located by any node in the system based on the node performing the placement algorithm for the required data.

Patent Agency Ranking