Abstract:
Techniques for performing database operations using vectorized instructions are provided. In one technique, it is determined whether to perform a database operation using one or more vectorized instructions or without using any vectorized instructions. This determination may comprise estimating a first cost of performing the database operation using one or more vectorized instructions and estimating a second cost of performing the database operation without using any vectorized instructions. Multiple factors that may be used to determine which approach to follow, such as the number of data elements that may fit into a SIMD register, a number of vectorized instructions in the vectorized approach, a number of data movement instructions that involve moving data from a SIMD register to a non-SIMD register and/or vice versa, a size of a cache, and a projected size of a hash table.
Abstract:
Techniques are provided herein for generating an integral image of an input image in parallel across the cores of a multi-core processor. The input image is split into a plurality of tiles, each of which is stored in a scratchpad memory associated with a distinct core. At each tile, a partial integral image of the tile is first computed over the tile, using a Single-Pass Algorithm. This is followed by aggregating partial sums belonging to subsets of tiles using a 2D Inclusive Parallel Prefix Algorithm. A summation is finally performed over the aggregated partial sums to generate the integral image over the entire input image.
Abstract:
A method for distributing tables to a cluster of nodes managed by database management system (DBMS), is disclosed. Multiple data placement schemes are evaluated based on a query workload set to select a data placement scheme for the cluster of nodes. Tables, used in join operations in the workload set, are selected for evaluation of data placement schemes. Query execution costs for the workload set are generated based on estimating a query execution cost for each data placement scheme for the tables. The data placement scheme that has least costly estimated execution cost for the workload set is selected as the data placement scheme for the cluster of nodes managed by DBMS.
Abstract:
A method and apparatus for processing path-based database operations is provided. According to one aspect, a path cache is maintained. For each hierarchical node that is traversed during a path-determining operation, it is determined whether a cache entry corresponding to that node is already contained in the path cache. If such a cache entry is already contained in the path cache, then the path indicated in that cache entry is used to complete the pathname for the node for which the operation is being performed. As a result, hierarchically higher nodes do not need to be traversed to complete the operation. Alternatively, if such a cache entry is not already contained in the path cache, then a cache entry for the node currently being traversed is generated and inserted into the path cache for use in subsequent path-determining operations.
Abstract:
Herein is described a data placement scheme for a distributed query processing systems that achieves load balance amongst the nodes of the system. To identify a node on which to place particular data, a supervisor node performs a placement algorithm over the particular data's identifier, where the placement algorithm utilizes two or more hash functions. The supervisor node runs the placement algorithm until a destination node is identified that is available to store the data, or the supervisor node has run the placement algorithm an established number of times. If no available node is identified using the placement algorithm, then an available destination node is identified for the particular data and information identifying the data and the selected destination node is included in an exception map. Most data may be located by any node in the system based on the node performing the placement algorithm for the required data.
Abstract:
Techniques for performing database operations using vectorized instructions are provided. In one technique, it is determined whether to perform a database operation using one or more vectorized instructions or without using any vectorized instructions. This determination may comprise estimating a first cost of performing the database operation using one or more vectorized instructions and estimating a second cost of performing the database operation without using any vectorized instructions. Multiple factors that may be used to determine which approach to follow, such as the number of data elements that may fit into a SIMD register, a number of vectorized instructions in the vectorized approach, a number of data movement instructions that involve moving data from a SIMD register to a non-SIMD register and/or vice versa, a size of a cache, and a projected size of a hash table.
Abstract:
A method, apparatus, and system for efficiently re-partitioning data using scheduled network communication are provided. Given re-partitioning data defining the data blocks to be sent amongst a plurality of server nodes, a corresponding network schedule is determined to send the data blocks in a coordinated manner. The network schedule is divided into time slots, wherein each of the plurality of server nodes can send up to one data block and receive up to one data block in each time slot. By using a greedy selection algorithm that prioritizes by largest senders and largest receivers, a near optimal schedule can be determined even in the presence of heavy skew. The greedy selection algorithm can be implemented with a O(T*N^2) time complexity, enabling scaling to large multi-node clusters with many server nodes. The network schedule is of particular interest for database execution plans requiring re-partitioning on operators with different keys.
Abstract:
Techniques for performing database operations using vectorized instructions are provided. In one technique, data compaction is performed using vectorized instructions to identify a shuffle mask based on matching bits and update an output array based on the shuffle mask and an input array. In a related technique, a hash table probe involves using vectorized instructions to determine whether each key in one or more hash buckets matches a particular input key.
Abstract:
A method for storing XML documents a hybrid navigation/streaming format is provided to allow efficient storage and processing of queries on the XML data that provides the benefits of both navigation and streaming and ameliorates the disadvantages of each. Each XML document to be stored is independently analyzed to determine a combination of navigable and streamable storage format that optimizes the processing of the data for anticipated access patterns.
Abstract:
A system and method for performing a query operation on a pair of relations in a database system coupled to a heterogeneous system (HS) is disclosed. Assuming that that pair of relations is partitioned and already loaded into the HS, the database system receives a query on the pair of relations and based on the type of query operation computes the cost of performing the query operation on the database alone or the costs of performing the query operation with the assistance of the HS, each of the costs corresponding to a particular algorithm. If the costs indicate that the HS improves the performance of the query operation, then the HS computes portions of the operation, and returns the results back to the database system. If any parts of the relation are out of sync with the database system, the database system performs operations to maintain transactional consistency.