摘要:
An inherently serial program is processed in parallel, thus leading to higher processing speeds, while maintaining a close approximation to the specific result obtained through a serial running of the program. This goal has been attained based on the fact that the desired degree of closeness between a parallel result and the serial result depends on the particular inherently serial program being run and the type of analysis being performed. That is, some inherently serial processes require a “fine-tuned” result while for others a “coarser” result is acceptable. The frequency at which the parallel branches consolidate their respective results is changed accordingly to alter the degree of closeness between the parallel processed result and the serially processed result.
摘要:
A computerized method and system for analyzing a multitude of items in a high dimensional (n-dimensional) data space Dn each described by n item features. The method uses a mining function f with at least one control parameter Pi controlling the target of the data mining function. The method selects a transformation function T for reducing dimensions of the n-dimensional space by space-filling curves mapping said n-dimensional space to a m-dimensional space (m
摘要:
Methods and apparatus, including computer program products, implementing and using techniques for compressing data included in several transactions. Each transaction has at least one item. A unique identifier is assigned to each different item and, if taxonomy is defined, to each different taxonomy parent. Sets of transactions are formed from the several transactions. The sets of transactions are stored using a computer data structure including: a list of identifiers of different items in the set of transactions, information indicating number of identifiers in the list, and bit field information indicating presence of the different items in the set of transactions, said bit field information being organized in accordance with the list for facilitating evaluation of patterns with respect to the set of transactions. A data structure for compressing data included in a set of transactions is also provided.
摘要:
A first data mining model and a second data mining model are compared. A first data mining model M1 represents results of a first data mining task on a first data set D1 and provides a set of first prediction values. A second data mining model M2 represents results of a second data mining task on a second data set D2 and provides a set of second prediction values. A relation R is determined between said sets of prediction values. For at least a first record of an input data set, a first and second probability distribution is created based on the first and second data mining models applied to the first record. A distance measure d is calculated for said first record using the first and second probability distributions and the relation. At least one region of interest is determined based on said distance measure d.
摘要:
A predictive analysis generates a predictive model (Padj(Y|X)) based on two separate pieces of information, a set of original training data (Dorig), and a “true” distribution of indicators (Ptrue(X)). The predictive analysis begins by generating a base model distribution (Pgen(Y|X)) from the original training data set (Dorig) containing tuples (x,y) of indicators (x) and corresponding labels (y). Using the “true” distribution (Ptrue(X)) of indicators, a random data set (D′) of indicator records (x) is generated reflecting this “true” distribution (Ptrue(X)). Subsequently, the base model (Pgen(Y|X)) is applied to said random data set (D′), thus assigning a label (y) or a distribution of labels to each indicator record (x) in said random data set (D′) and generating an adjusted training set (Dadj). Finally, an adjusted predictive model (Padj(Y|X)) is trained based on said adjusted training set (Dadj).
摘要:
The invention relates to a method for determining a time for retraining a data mining model, including the steps of: calculating multivariate statistics of a training model during a training phase; storing the multivariate statistics in the data mining model; evaluating reliability of the data mining model based on the multivariate statistics and at least one distribution parameter, and deciding to retrain the data mining model based on an arbitrary measure of one or more statistical parameters including an F-test statistical analysis.
摘要:
Methods and apparatus, including computer program products, implementing and using techniques for pattern detection in input data containing several transactions, each transaction having at least one item. Filter conditions for interesting patterns are received, and a first set of filter conditions applicable in connection with generation of candidate patterns is determined. An evaluated candidate pattern is selected as a parent candidate pattern, and evaluation information about the parent candidate pattern is maintained. Child candidate patterns are generated by extending the parent candidate pattern and taking into account the first set of filter conditions. The child candidate patterns are evaluated with respect to the input data together in sets of similar candidate patterns and based on the evaluation information about the parent candidate pattern. At least one child candidate pattern successfully passing the evaluation step is recursively used as a parent candidate pattern.
摘要:
A process of transforming data residing in databases, such as relational databases, into forms suitable as input to data analysis tools, such as predictive modeling tools includes the steps of defining a business process problem to be solved and identifying data requirements. For example, the business process problem may relate to predicting a customer's propensity to make purchases in the future or a store's requirements for inventory in the future. In the process, a computer implemented method is used for automatically transforming data for data analysis such as predictive modeling. Database metadata that describe database tables, their interrelationships, dimensional information, fact tables and measures are accessed. A mining transformation profile is created to encapsulate aggregations and transformation on data stored in relational databases in order to convert the data to forms suitable for predictive mining tools. The mining transformation profile specifies data transformations relative to the data base metadata. Executable data transformation codes is then generated from the database metadata and the mining transformation profile. Execution of this code results in aggregation and transformation of data residing in a database for input to a data analysis tool such as a predictive modeling tool. The data transformation code can be used by, for example, the predictive modeling tool to generate an output that provides a solution to a business process problem.
摘要:
Methods and apparatus, including computer program products, implementing and using techniques for compressing data included in several transactions. Each transaction has at least one item. A unique identifier is assigned to each different item and, if taxonomy is defined, to each different taxonomy parent. Sets of transactions are formed from the several transactions. The sets of transactions are stored using a computer data structure including: a list of identifiers of different items in the set of transactions, information indicating number of identifiers in the list, and bit field information indicating presence of the different items in the set of transactions, said bit field information being organized in accordance with the list for facilitating evaluation of patterns with respect to the set of transactions. A data structure for compressing data included in a set of transactions is also provided.
摘要:
Embodiments of the invention provide a method for detecting changes in behavior of authorized users of computer resources and reporting the detected changes to the relevant individuals. The method includes evaluating actions performed by each user against user behavioral models and business rules. As a result of the analysis, a subset of users may be identified and reported as having unusual or suspicious behavior. In response, the management may provide feedback indicating that the user behavior is due to the normal expected business needs or that the behavior warrants further review. The management feedback is available for use by machine learning algorithms to improve the analysis of user actions over time. Consequently, investigation of user actions regarding computer resources is facilitated and data loss is prevented more efficiently relative to the prior art approaches with only minimal disruption to the ongoing business processes.