Apparatus and Method for Query Prioritization in a Shared Nothing Distributed Database
    1.
    发明申请
    Apparatus and Method for Query Prioritization in a Shared Nothing Distributed Database 有权
    无共享分布式数据库中查询优先级的设备和方法

    公开(公告)号:US20110246481A1

    公开(公告)日:2011-10-06

    申请号:US12751208

    申请日:2010-03-31

    IPC分类号: G06F17/30 G06F15/173

    摘要: A shared nothing distributed database system includes a master node and a set of shared nothing nodes. Each shared nothing node includes a node state table stored in memory that characterizes various statements and multiple processes implementing each statement, target resource consumption rate for each process and process activity information. A monitor module executed by a central processing unit processes information in the node state table and adjusts values in the node state table in accordance with priority criteria specifying individual user priority ascribed to a statement. A query associated with a statement is processed in the shared nothing distributed database system in accordance with the priority criteria.

    摘要翻译: 无共享的分布式数据库系统包括主节点和一组无共享节点。 每个无共享节点包括存储在存储器中的节点状态表,其表征各种语句和实现每个语句的多个进程,每个进程的目标资源消耗速率和进程活动信息。 由中央处理单元执行的监视器模块处理节点状态表中的信息,并且根据指定归因于语句的个人用户优先级的优先级标准来调整节点状态表中的值。 根据优先级标准,在共享无分布式数据库系统中处理与语句关联的查询。

    MEMORY QUOTA
    2.
    发明申请

    公开(公告)号:US20120239641A1

    公开(公告)日:2012-09-20

    申请号:US13051986

    申请日:2011-03-18

    IPC分类号: G06F17/30

    CPC分类号: G06F17/30486 G06F12/08

    摘要: A method, article of manufacture, and apparatus for processing queries, comprising analyzing a query tree, determining at least one operator based on the query tree analysis, assigning a memory allocation for each of the at least one operator, and storing the assignment in a storage device. In some embodiments, a memory classification for each of the at least one operator is determined. In some embodiments, assigning a memory allocation for each of the at least one operator includes assigning a memory allocation based on the memory classification.

    摘要翻译: 一种用于处理查询的方法,制品和装置,包括分析查询树,基于所述查询树分析确定至少一个运算符,为所述至少一个运算符中的每一者分配存储器分配,以及将所述分配存储在 储存设备。 在一些实施例中,确定至少一个操作者中的每一个的存储器分类。 在一些实施例中,为所述至少一个操作者中的每一个分配内存分配器包括基于所述存储器分类分配存储器分配。

    Memory quota
    4.
    发明授权
    Memory quota 有权
    内存配额

    公开(公告)号:US09418109B2

    公开(公告)日:2016-08-16

    申请号:US13051986

    申请日:2011-03-18

    IPC分类号: G06F17/30 G06F12/08

    CPC分类号: G06F17/30486 G06F12/08

    摘要: A method, article of manufacture, and apparatus for processing queries, comprising analyzing a query tree, determining at least one operator based on the query tree analysis, assigning a memory allocation for each of the at least one operator, and storing the assignment in a storage device. In some embodiments, a memory classification for each of the at least one operator is determined. In some embodiments, assigning a memory allocation for each of the at least one operator includes assigning a memory allocation based on the memory classification.

    摘要翻译: 一种用于处理查询的方法,制品和装置,包括分析查询树,基于所述查询树分析确定至少一个运算符,为所述至少一个运算符中的每一者分配存储器分配,以及将所述分配存储在 储存设备。 在一些实施例中,确定至少一个操作者中的每一个的存储器分类。 在一些实施例中,为所述至少一个操作者中的每一个分配内存分配器包括基于所述存储器分类分配存储器分配。

    Executing correlated and multi-row subqueries in a MPP database
    5.
    发明授权
    Executing correlated and multi-row subqueries in a MPP database 有权
    在MPP数据库中执行相关和多行子查询

    公开(公告)号:US08862606B1

    公开(公告)日:2014-10-14

    申请号:US13240923

    申请日:2011-09-22

    IPC分类号: G06F17/30

    CPC分类号: G06F17/30545

    摘要: The invention enables a correlated or multi-row subquery (CSQ) to be performed on distributed MPP and shared-nothing databases by broadcasting intermediate results, prior to a correlation operation, from subquery execution on one segment to all other segments in the distributed database so that the respective CSQs of each segment will have access to the necessary results to permit correct execution of the CSQ. Additionally, the intermediate results are saved to disk to avoid the necessity of replicating the same intermediate results multiple times during execution of a subquery plan.

    摘要翻译: 本发明使得能够在分布式MPP和无共享数据库上执行相关或多行子查询(CSQ),在相关操作之前,从分段数据库中的一个段上的子查询执行到所有其他段的广播中间结果, 每个分段的相应CSQ将获得必要的结果,以便正确执行CSQ。 另外,将中间结果保存到磁盘,以避免在执行子查询计划期间多次复制相同的中间结果。

    Systems and methods for auto-scaling a big data system

    公开(公告)号:US11474874B2

    公开(公告)日:2022-10-18

    申请号:US14459631

    申请日:2014-08-14

    IPC分类号: G06F9/50

    摘要: Systems and methods for automatically scaling a big data system. Methods include determining, at a first time, a first number of nodes for a cluster to process a request; assigning an amount of nodes equal to the first number of nodes to the cluster; determining a rate of progress of the request; determining, at a second time based on the rate of progress a second number of nodes; and modifying the amount of nodes to equal the second number of nodes. Systems include a cluster manager, to add and/or remove any nodes; the big data system, to process requests that utilize the cluster and nodes, and an automatic scaling cluster manager including a big data interface for communicating with the big data system; a cluster manager interface for communicating with the cluster manager; and a cluster state machine.

    Resource queues
    8.
    发明授权
    Resource queues 有权
    资源队列

    公开(公告)号:US09418064B1

    公开(公告)日:2016-08-16

    申请号:US13051980

    申请日:2011-03-18

    IPC分类号: G06F17/30 G06F15/173

    摘要: A method, article of manufacture, and apparatus for processing queries, comprising receiving a query, determining a query classification for the query, assigning the query to a resource queue based on the determined query classification, and placing the query in the assigned resource queue. In some embodiments, the resource queue is divided into a plurality of slots, and the query is placed in a slot. The resource queue may be associated with a resource queue memory allocation, and each of the plurality of slots is associated with a slot memory allocation.

    摘要翻译: 一种用于处理查询的方法,制品和装置,包括接收查询,确定所述查询的查询分类,基于所确定的查询分类将所述查询分配给资源队列,以及将所述查询放置在所分配的资源队列中。 在一些实施例中,资源队列被划分成多个时隙,并且查询被放置在时隙中。 资源队列可以与资源队列存储器分配相关联,并且多个时隙中的每一个与时隙存储器分配相关联。

    Systems and Methods for Auto-Scaling a Big Data System
    9.
    发明申请
    Systems and Methods for Auto-Scaling a Big Data System 审中-公开
    自动缩放大数据系统的系统和方法

    公开(公告)号:US20160048415A1

    公开(公告)日:2016-02-18

    申请号:US14459631

    申请日:2014-08-14

    IPC分类号: G06F9/50

    CPC分类号: G06F9/5083 G06F9/5072

    摘要: Systems and methods for automatically scaling a big data system are disclosed. Methods may include: determining, at a first time, a first optimal number of nodes for a cluster to adequately process a request; assigning an amount of nodes equal to the first optimal number; determining a rate of progress of the request; determining, at a second time based on the rate of progress a second optimal number of nodes; and modifying the number of nodes assigned to the cluster to equal the second optimal number. Systems may include: a cluster manager, to add and/or remove nodes; a big data system, to process requests that utilize the cluster and nodes, and an automatic scaling cluster manager, including: a big data interface, for communicating with the big data system; a cluster manager interface, for communicating with a cluster manager instructions for adding and/or removing nodes from a cluster used to process a request; and a cluster state machine.

    摘要翻译: 公开了用于自动缩放大数据系统的系统和方法。 方法可以包括:在第一时间确定簇的第一最佳数量的节点以充分地处理请求; 分配等于第一最优数的节点数; 确定请求的进度; 基于所述进展速度在第二时间确定第二最佳数量的节点; 以及修改分配给所述集群的节点数量等于所述第二最佳数量。 系统可以包括:群集管理器,用于添加和/或移除节点; 一个大数据系统,用于处理利用集群和节点的请求,以及一个自动扩展集群管理器,包括:一个大数据接口,用于与大数据系统进行通信; 集群管理器接口,用于与集群管理器通信用于从用于处理请求的集群中添加和/或移除节点的指令; 和集群状态机。

    RLE-aware optimization of SQL queries
    10.
    发明授权
    RLE-aware optimization of SQL queries 有权
    RLE感知SQL查询优化

    公开(公告)号:US09171041B1

    公开(公告)日:2015-10-27

    申请号:US13248536

    申请日:2011-09-29

    IPC分类号: G06F17/30

    摘要: SQL queries are optimized to operate directly on compressed data (and obtain the correct result) rather than requiring that the data be first decompressed prior to processing a query. Certain characteristic pattern trees are mapped against a logical input query plan that includes certain logical operators such as a DECOMPRESS that precedes a JOIN or a GROUPBY in association with a COUNT to identify instances in the plan that match a characteristic pattern. Upon locating a match, the input query plan is transformed into a logically equivalent plan that operates correctly on compressed data, by analyzing the interplay of the semantics of logical query operations with the compressed data and substituting less costly structures and operations. DECOMPRESS operations are moved to operate subsequent to a JOIN or eliminated altogether, and COUNT operations are replaced by a different operation, such as SUM, that is logically equivalent for compressed data.

    摘要翻译: SQL查询经过优化,可直接对压缩数据进行操作(并获得正确的结果),而不是要求数据在处理查询之前先进行解压缩。 某些特征模式树被映射到逻辑输入查询计划,逻辑输入查询计划包括某些逻辑运算符,例如在与COUNT相关联的JOIN或GROUPBY之前的DECOMPRESS,以识别计划中匹配特征模式的实例。 在找到匹配后,通过分析逻辑查询操作的语义与压缩数据的相互作用并代替较低成本的结构和操作,将输入查询计划转换为在压缩数据上正确操作的逻辑等效计划。 DECOMPRESS操作被移动到在JOIN之后操作或完全消除,并且COUNT操作被诸如SUM之类的不同操作所替代,在逻辑上等同于压缩数据。