Abstract:
An approach for implementing function semantic based partition-wise SQL execution and partition pruning in a data processing system is provided. The system receives a query directed to a range-partitioned table and determines if operation key(s) of the query include function(s) over the table partitioning key(s). If so, the system obtains a set of values corresponding to each partition by evaluating the function(s) on a low bound and/or a high bound table partitioning key value corresponding to the partition. The system may then compare the sets of values corresponding to different partitions and determine whether to aggregate results obtained by executing the query over the partitions based on the comparison. The system may also determine whether to prune any partitions from processing based on a set of correlations between the set of values for each partition and predicate(s) of the query including function(s) over the table partitioning key(s).
Abstract:
Techniques for creating and using partial indexes are provided. A partial index is an index that indexes one or more partitions of a partitioned table and does not index one or more other partitions of the partitioned table. Thus, if a partition of a partitioned table is rarely used, then an index may index all other partitions of the partitioned table or at least only those partitions that are written to or read from relatively frequently. Also, in preparation for loading data into a partition, indexing for the partition may be “turned off”, effectively making a full index a partial index. While the data is loaded into the partition, the partial index is still available for query processing. After the data is loaded into the partition, indexing for the partition is “turned on”, which causes the data in the partition to be indexed by the partial index.
Abstract:
Techniques are provided for moving data between partitions. Such a process may be performed without requiring any locks that block transactions that target a partition from being executed. Instead, such transactions may proceed while a move operation is being performed. The move operation involves copying data from the targeted partition to another partition that is hidden from (or “invisible” to) those transactions that attempt to read from or write to the partition. During the move operation, changes that are made to the partition are also reflected in a journal. Eventually, the changes reflected in the journal are drained and applied to the hidden partition. Once the partition and the hidden partition are synchronized, the identities of the partitions are swapped so that future transactions will target the previously-hidden partition instead of the previously-viewable partition.
Abstract:
Techniques for processing “destructive” database statements are provided. Destructive database statements, when processed, cause metadata of a database object to be changed. Examples of such database statements include ones that delete an index, that set a column as unused, and that drop a constraint on a column. When such a statement is received, a change is made to metadata of a database object. Such a metadata change may involve setting an index as unusable, disabling a constraint, or invalidating a cursor. After the metadata change, a first time is determined. Then, it is determined when one or more database transactions that were pending at the first time have committed. After those database transaction(s) have committed, one or more operations are performed, such as dropping an index or dropping a constraint.
Abstract:
Techniques are provided for moving data between partitions. Such a process may be performed without requiring any locks that block transactions that target a partition from being executed. Instead, such transactions may proceed while a move operation is being performed. The move operation involves copying data from the targeted partition to another partition that is hidden from (or “invisible” to) those transactions that attempt to read from or write to the partition. During the move operation, changes that are made to the partition are also reflected in a journal. Eventually, the changes reflected in the journal are drained and applied to the hidden partition. Once the partition and the hidden partition are synchronized, the identities of the partitions are swapped so that future transactions will target the previously-hidden partition instead of the previously-viewable partition.
Abstract:
Systems, methods, and other embodiments associated with equivalence reasoning are described. One example method includes iteratively inputting batches of unprocessed equivalence pairs from a semantic model to an operating memory. In the operating memory, one or more cliques for the input batches are built until no further batches remain. A clique designates a canonical representative resource for a group of equivalent resources as determined from the equivalence pairs. The one or more cliques are built for the input batches to a clique map in a remote access memory. The clique map is returned for use by the semantic model.
Abstract:
Systems for managing large database systems. Storage devices hold a set of one or more subject database tables that hold subject data accessed by one or more users. A data definition statement to perform a table alteration operation (e.g., a table partition operation) over a particular table in the subject database is received, parsed, and the table alteration operation over the particular table is initiated. During the processing of the data definition statement to complete the table alteration operation, a data manipulation statement to perform a table access operation over the same table is received and parsed. The data manipulation statement is not blocked, even while the table alteration operations are ongoing. Rather, during the time that the table alteration operations are ongoing, the data manipulation statement is processed using hidden tables. After the table alteration operations have completed, incoming data manipulation statements are performed over the altered database tables.
Abstract:
Systems, methods, and other embodiments associated with data sources adapted for parallel inference on triples associated with a semantic model are described. One example method includes creating a source table that is partitioned on triple predicate and stores triples for entailment. The source table may store compact triple identifiers that have been mapped to triple identifiers from the semantic model.
Abstract:
What is disclosed is an improved approach to perform automatic partitioning, without requiring any expertise on the part of the user. A three stage processing pipeline is provided to generate candidate partition schemes, to evaluate the candidate using real table structures that are empty, and to then implement a selected scheme with production data for evaluation. In addition, an improved approach is described to perform automatic interval partitioning, where the inventive concept implements interval partitioning that does not impose these implicit constraints on the partition key column.
Abstract:
Disclosed is a system, method, and computer program product that allows multiple partition schemes to be applied to a single object. Each scheme represents the same set of partitions, but the schemes may have different partition methods, different partition keys, and/or different partition bounds.