ADAPTIVE DATA REPARTITIONING AND ADAPTIVE DATA REPLICATION
    1.
    发明申请
    ADAPTIVE DATA REPARTITIONING AND ADAPTIVE DATA REPLICATION 审中-公开
    自适应数据分配和自适应数据复制

    公开(公告)号:US20160253402A1

    公开(公告)日:2016-09-01

    申请号:US14634199

    申请日:2015-02-27

    CPC classification number: G06F17/30584

    Abstract: A method and apparatus for adaptive data repartitioning and adaptive data replication is provided. A data set stored in a distributed data processing system is partitioned by a first partitioning key. A live workload comprising a plurality of data processing commands is processed. While processing the live workload, statistical properties of the live workload are maintained. Based on the statistical properties of the live workload with respect to the data set, it is determined to replicate and/or repartition the data set by a second partitioning key. The replicated and/or repartitioned data set is partitioned by the second partitioning key.

    Abstract translation: 提供了一种用于自适应数据重新分配和自适应数据复制的方法和装置。 存储在分布式数据处理系统中的数据集由第一分区键划分。 处理包括多个数据处理命令的实时工作。 在处理实时工作负载时,维持实时工作负载的统计属性。 基于相对于数据集的实时工作负载的统计特性,确定通过第二分区密钥复制和/或重新分配数据集。 复制和/或重新分区的数据集由第二分区密钥分隔。

    Application-level dynamic scheduling of network communication for efficient re-partitioning of skewed data

    公开(公告)号:US10263893B2

    公开(公告)日:2019-04-16

    申请号:US15372224

    申请日:2016-12-07

    Abstract: Techniques are provided for using decentralized lock synchronization to increase network throughput. In an embodiment, a first computer sends, to a second computer comprising a lock, a request to acquire the lock. In response to receiving the lock acquisition request, the second computer detects whether the lock is available. If the lock is unavailable, then the second computer replies by sending a denial to the first computer. Otherwise, the second computer sends an exclusive grant of the lock to the first computer. While the first computer has acquired the lock, the first computer sends data to the second computer. Afterwards, the first computer sends a request to release the lock to the second computer. This completes one duty cycle of the lock, and the lock is again available for acquisition.

    Adaptive resolution hsitogram
    5.
    发明授权

    公开(公告)号:US10146806B2

    公开(公告)日:2018-12-04

    申请号:US14621204

    申请日:2015-02-12

    Abstract: A method, apparatus, and system for determining a data distribution is provided by using an adaptive resolution histogram. In an embodiment, the adaptive resolution histogram is created using a trie, wherein node values in the trie represent frequency distributions and node positions define associated keys or key prefixes. Keys are derived from input data such as database records that are streamed from a record source. These keys may be processed as received to build the trie in parallel with the production of the input data. To provide adaptive resolution, new child nodes may only be created in the trie when a node value is incremented beyond a predetermined threshold. In this manner, the histogram adjusts the allocation of nodes according to the actual distribution of the data. The completed adaptive resolution histogram may be used for various tasks such as partitioning for balanced parallel processing of the input data.

Patent Agency Ranking