Abstract:
Methods and apparatus, including computer program products, for compression of tables based on occurrence of values. In general, a number representing an amount of occurrences of a frequently occurring value in a group of adjacent rows of a column is generated, a vector representing whether the frequently occurring value exists in a row of the column is generated, and the number and the vector are stored to enable searches of the data represented by the number and the vector. The vector may omit a portion representing the group of adjacent rows. The values may be dictionary-based compression values representing business data such as business objects. The compression may be performed in-memory, in parallel, to improve memory utilization, network bandwidth consumption, and processing performance.
Abstract:
A query having multiple parts may be processed to form an intermediate results set. This intermediate results set may be partitioned into a plurality of groups. Thereafter, the groups may be sorted into a plurality of containers so that each container contains data sufficient to calculate one requested result in the multipart query. Related techniques, apparatuses, systems, and computer program products are also described.
Abstract:
In business systems, one or more methods can be used to reduce an amount of redundant data. In one implementation, a method to reduce redundancy within a data model in a database, in which the data model is represented by at least one table, includes determining a number of distinct values of partial keys in a table. Each partial key represents at least one row in the table. The method includes reordering one or more columns of the table by cardinality of partial keys, in which the cardinality of a partial key represents a number of distinct values of the partial key. The method further includes determining whether pairs of partial keys are functionally dependent and eliminating one or more columns having functional dependencies from the table.
Abstract:
Methods and apparatus, including computer systems and program products, for executing a query on a subset of data, for example, to facilitate a fast search with a very large result set. In one general aspect, a method of executing a query includes receiving a query for execution on data in the data repository; generating an estimate of a number of results of the query; defining a subset of data in the data repository; determining whether to execute the query on the subset of the data; executing the query on the subset of the data to generate a partial set of results if the query is to be executed on the subset of the data, otherwise executing the query on the data repository to generate a complete set of results; and providing query results.
Abstract:
Methods and apparatus, including computer systems and program products, relating to an information management system and aggregating data by performing table scans. In general, in one aspect, the technique includes receiving a query for a response to a search on a database, loading data from the database into memory, filtering the data based on the query to generate a list of results, buffering at least one key figure corresponding to a result, buffering at least one dimension value corresponding to each key figure, aggregating the dimension values to generate an aggregate key, aggregating key figures corresponding to the sane aggregate key to generate one or more aggregate key figures, and displaying the response to the search on a display device. Loading the data may include compressing the data. Filtering the data may be performed blockwise.
Abstract:
A system and method for executing a query plan are disclosed. In the system and method, a join graph is generated to represent the query plan. The join graph includes a set of plan operations that are to be executed for implementing the join graph. The query plan is received by a distributed network of a logical index server and one or more selected physical index servers. Each physical index server receives a portion of the plan operations, and determines what plan data is needed to execute the portion of the plan operations. A system and method includes a process for determining what plan data is needed from other physical index servers, or what plan data is needed by other physical index servers.
Abstract:
In business systems, one or more methods can be used to reduce an amount of redundant data. In one implementation, a method to reduce redundancy within a data model in a database, in which the data model is represented by at least one table, includes determining a number of distinct values of partial keys in a table. Each partial key represents at least one row in the table. The method includes reordering one or more columns of the table by cardinality of partial keys, in which the cardinality of a partial key represents a number of distinct values of the partial key. The method further includes determining whether pairs of partial keys are functionally dependent and eliminating one or more columns having functional dependencies from the table.
Abstract:
The subject matter disclosed herein provides methods for the dual storage of data using an in-memory array and an on-disk page structure. An in-memory array holding a column of data can be maintained. One or more pages can be maintained. Each of the one or more pages can have one or more rows for storing the column of data. Random access can be provided to a subset of the one or more rows by at least loading the subset of rows from the one or more pages to the in-memory array without loading all of the rows from the one or more pages. Related apparatus, systems, techniques, and articles are also described.
Abstract:
Methods and apparatus, including computer program products, for block compression of tables with repeated values. In general, value identifiers representing a compressed column of data may be sorted to render repeated values contiguous, and block dictionaries may be generated. A block dictionary may be generated for each block of value identifiers. Each block dictionary may include a list of block identifiers, where each block identifier is associated with a value identifier and there is a block identifier for each unique value in a block. Blocks may have standard sizes and block dictionaries may be reused for multiple blocks.
Abstract:
Methods and apparatus, including computer program products, for block compression of tables with repeated values. In general, value identifiers representing a compressed column of data may be sorted to render repeated values contiguous, and block dictionaries may be generated. A block dictionary may be generated for each block of value identifiers. Each block dictionary may include a list of block identifiers, where each block identifier is associated with a value identifier and there is a block identifier for each unique value in a block. Blocks may have standard sizes and block dictionaries may be reused for multiple blocks.