摘要:
Provided are a method, system, and article of manufacture for discovering transformations applied to a source table to generate a target table. Selection is made of a source table comprising a plurality of rows and a target table resulting from a transformation applied to the rows of the source table. A first pre-processing method is applied with respect to columns in the source and target tables to produce first category pre-processing output. The first category pre-processing output is used to determine first category transformation rules with respect to at least one source table column and at least one target table column. For each unpredicted target column in the target table not predicted by the determined first category transformation rules, a second pre-processing method is applied to columns in the source table and unpredicted target columns to produce second category pre-processing output. The second category pre-processing output is used to determine second category transformation rules with respect to at least one source table column and at least one target table column.
摘要:
Provided are a method, system, and article of manufacture for discovering transformations applied to a source table to generate a target table. Selection is made of a source table comprising a plurality of rows and a target table resulting from a transformation applied to the rows of the source table. A first pre-processing method is applied with respect to columns in the source and target tables to produce first category pre-processing output. The first category pre-processing output is used to determine first category transformation rules with respect to at least one source table column and at least one target table column. For each unpredicted target column in the target table not predicted by the determined first category transformation rules, a second pre-processing method is applied to columns in the source table and unpredicted target columns to produce second category pre-processing output. The second category pre-processing output is used to determine second category transformation rules with respect to at least one source table column and at least one target table column.
摘要:
Provided are a method, system, and article of manufacture for using a data mining algorithm to discover data rules. A data set including multiple records is processed to generate data rules for the data set. Each record has a record format including a plurality of fields and each rule provides a predicted condition for one field based on at least one predictor condition in at least one other field. The generated data rules are provided to a user interface to enable a user to edit the generated data rules. The data rules are stored in a rule repository to be available to use to validate data sets having the record format.
摘要:
Provided are an article of manufacture, system, and method for using a data mining algorithm to generate rules used to validate a selected region of a predicted column. A data set has a plurality of columns and records providing data for each of the columns. Selection is received of at least one predicted column for which rules are to be generated and at least one region of the selected at least one predicted column, wherein each region specifies data positions in the column. The data set is processed to determine association relationships among data in at least one predictor column and subsequences in the selected at least one region of the at least one predicted column. At least one rule is generated from the relationships specifying a condition involving at least one predictor column that predicts at least one value in the selected region of the at least one predicted column.
摘要:
Provided are a method, system, and article of manufacture for using a data mining algorithm to generate format rules used to validate data sets. A data set has a plurality of columns and records providing data for each of the columns. Selection is received of at least one format column for which format rules are to be generated and selection is received of at least one predictor column. A format mask column is generated for each selected format column. For records in the data set, a value in the at least one format column is converted to a format mask representing a format of the value in the format column and storing the format mask in the format mask column in the record for which the format mask was generated. The at least one predictor column and the at least one format mask column are processed to generate at least one format rule. Each format rule specifies a format mask associated with at least one condition in the at least one predictor column.
摘要:
Provided are a method, system, and article of manufacture for using a data mining algorithm to generate format rules used to validate data sets. A data set has a plurality of columns and records providing data for each of the columns. Selection is received of at least one format column for which format rules are to be generated and selection is received of at least one predictor column. A format mask column is generated for each selected format column. For records in the data set, a value in the at least one format column is converted to a format mask representing a format of the value in the format column and storing the format mask in the format mask column in the record for which the format mask was generated. The at least one predictor column and the at least one format mask column are processed to generate at least one format rule. Each format rule specifies a format mask associated with at least one condition in the at least one predictor column.
摘要:
Provided are a method, system, and article of manufacture for managing validation models and rules to apply to data sets. A schema definition describing a structure of at least one column in a first data set having a plurality of columns and records providing data for each of the columns is received. At least one model is generated, wherein each model asserts conditions for at least one column in a record of the first data set. The schema definition and the at least one model are stored in a data quality model. Selection is received of a second data set and the data quality model. A determination is made as to whether a structure of the second data set is compatible with the schema definition in the selected data quality model. Each model in the data quality model is applied to the records in the second data set to validate the records in the second data set in response to determining that the structure of the second data set and the schema definition are compatible.
摘要:
Various embodiments of a method, system and article of manufacture to discover relationships among a first set of elements and a second set of elements are provided. At least one metric algorithm is identified based on a metric selection parameter. A raw result is determined based on the at least one metric algorithm, a first specified structural description of the first set of elements and a second specified structural description of the second set of elements. The raw result comprises a plurality of relationship measurements and the raw result is ordered. In some embodiments, a balanced result is produced based on the raw result and a matching strategy algorithm. In other embodiments, the matching strategy algorithm is identified based on a matching strategy selection parameter.
摘要:
A system and method for the dynamic categorization of heterogeneous, regulated enterprise information assets. In one embodiment of the invention a system includes a computer network controlled by an enterprise and a database including a plurality of enterprise data entities. A user interface, through which a plurality of enterprise users may access the enterprise data entities, is also used by the plurality of users to assign user-defined categories to the enterprise data entities. The user interface is configured to enable a plurality of the users to access and assign additional user-defined categories to enterprise data entities having user-defined categories previously assigned by other users.
摘要:
Various embodiments of a system and computer program product to access metadata from a plurality of data servers from a federated database management system are provided. In one embodiment, a request for metadata, from a client application, is received by the federated database management system. Data servers which are accessible from the federated database management system are identified. For each data server, metadata describing data of a data source of that data server is retrieved in accordance with the application request. The retrieved metadata from each of the data servers is aggregated to produce an aggregated result in a uniform format. The aggregated result is provided. In another embodiment, for each data server, a source metadata request for metadata of that data server is generated in accordance with the application request and a source metadata application programming interface. A view is created based on the source metadata request for metadata for each data server.