Abstract:
Embodiments of the present invention provide for batch and incremental loading of data into a database. In the present invention, the loader infrastructure utilizes machine code database instructions and hardware acceleration to parallelize the load operations with the I/O operations. A large, hardware accelerator memory is used as staging cache for the load process. The load process also comprises an index profiling phase that enables balanced partitioning of the created indexes to allow for pipelined load. The online incremental loading process may also be performed while serving queries.
Abstract:
Embodiments of the present invention provide hardware-friendly indexing of databases. In particular, forward and reverse indexing are utilized to allow for easy traversal of primary key to foreign key relationships. A novel structure known as a hit list also allows for easy scanning of various indexes in hardware. Group indexing is provided for flexible support of complex group key definition, such as for date range indexing and text indexing. A Replicated Reordered Column (RRC) may also be added to the group index to convert random I/O pattern into sequential I/O of only needed column elements.
Abstract:
Systems, architectures, and data structures are described which are used to manage distributed design chains, specifically for domains in which data reside in multiple applications and are linked through complex interrelationships. The design chains or design networks integrated by the invention may include multiple companies in multiple sites collaborating to design and develop a new product. The invention is intended to integrate seamlessly and transparently with existing, diverse legacy applications, which include inter-linked data relevant to the design, thereby addressing the needs identified above.
Abstract:
A PC-type computer has a system bus (e.g., a PCI bus) configured with a main CPU board, a statistical multiplexing (stat-mux) board, and a plurality of video/audio encoder boards, each configured to receive and compress a corresponding video/audio stream. The stat-mux board performs statistical multiplexing on the different compressed bitstreams to transmit multiple bitstreams over individual shared communication channels. Although each of the boards is configured to the system bus, each encoder board has a digital signal processor (DSP) with a synchronized serial interface (SSI) output port that is directly connected to an SSI input port on a DSP on the stat-mux board (which, in one embodiment, has four such DSPs each with six such SSI input ports). As such, (up to 24) compressed video/audio bitstreams generated on the various encoder boards can be transmitted directly to the stat-mux board without having to go through the system bus. In this way, the computer system can provide statistical multiplexing of low-latency video/audio bitstreams without having to suffer the processing delays associated with conventional transmission over PCI system buses.
Abstract:
An apparatus and method for reducing memory resource requirements in, e.g., an image processing system by utilizing a packed data pixel representation and, optionally, M-ary pyramid decomposition, for pixel block or pixel group searching and matching operations.
Abstract:
During video coding, a transform such as a discrete cosine transform (DCT) is applied to blocks of image data (e.g., motion-compensated interframe pixel differences) and the resulting transform coefficients for each block are quantized at a specified quantization level. Notwithstanding the fact that some coefficients are quantized to non-zero values, at least one non-zero quantized coefficient is treated as if it had a value of zero for purposes of further processing (e.g., run-length encoding (RLE) the quantized data). When segmentation analysis is performed to identify two or more different regions of interest in each frame, the number of coefficients that are treated as having a value of zero for RLE is different for different regions of interest (e.g., more coefficients for less-important regions). In this way, the number of bits used to encode image data are reduced to satisfy bit rate requirements without (1) having to drop frames adaptively, while (2) conforming to constraints that may be imposed on the magnitude of change in quantization level from frame to frame.
Abstract:
Embodiments of the present invention provide one or more hardware-friendly data structures that enable efficient hardware acceleration of database operations. In particular, the present invention employs a column-store format for the database. In the database, column-groups are stored with implicit row ids (RIDs) and a RID-to-primary key column having both column-store and row-store benefits via column hopping and a heap structure for adding new data. Fixed-width column compression allow for easy hardware database processing directly on the compressed data. A global database virtual address space is utilized that allows for arithmetic derivation of any physical address of the data regardless of its location. A word compression dictionary with token compare and sort index is also provided to allow for efficient hardware-based searching of text. A tuple reconstruction process is provided as well that allows hardware to reconstruct a row by stitching together data from multiple column groups.
Abstract:
Embodiments of the present invention provide one or more hardware-friendly data structures that enable efficient hardware acceleration of database operations. In particular, the present invention employs a column-store format for the database. In the database, column-groups are stored with implicit row ids (RIDs) and a RID-to-primary key column having both column-store and row-store benefits via column hopping and a heap structure for adding new data. Fixed-width column compression allow for easy hardware database processing directly on the compressed data. A global database virtual address space is utilized that allows for arithmetic derivation of any physical address of the data regardless of its location. A word compression dictionary with token compare and sort index is also provided to allow for efficient hardware-based searching of text. A tuple reconstruction process is provided as well that allows hardware to reconstruct a row by stitching together data from multiple column groups.
Abstract:
Embodiments of the present invention provide processing elements that are capable of performing high level database operations in hardware based on machine code instructions. These processing elements employ a dataflow architecture that operates on data in hardware without interruption or software. A scanning/indexing processing element may comprise logic that analyze database column groups stored in local memory, perform parallel field extraction and comparison, and generates a list of row pointers (row ids or RIDs) referencing those rows whose value(s) satisfy an applied predicate. The scanning/indexing processing may also be used to project database column groups, search and join index structures, and manipulate in-flight metadata flows, composing, merging, reducing, and modifying multi-dimensional lists of intermediate and final results. Furthermore, a scanning/indexing processing element may be used for joins with indexes, like a Group Index, which involves the association of each input tuple with potentially many related data components, in a one-to-many mapping. An XCAM processing element may comprise logic to perform associative database operations, like accumulation and aggregation, sieving, sorting and associative joins.
Abstract:
Systems, architectures, and data structures are described which are used to manage distributed design chains, specifically for domains in which data reside in multiple applications and are linked through complex interrelationships. The design chains or design networks integrated by the invention may include multiple companies in multiple sites collaborating to design and develop a new product. The invention is intended to integrate seamlessly and transparently with existing, diverse legacy applications, which include inter-linked data relevant to the design, thereby addressing the needs identified above.