公开(公告)号：US09081817B2

公开(公告)日：2015-07-14

申请号：US13084527

申请日：2011-04-11

申请人： Arvind Arasu ,  Michaela Götz ,  Shriraghav Kaushik

发明人： Arvind Arasu ,  Michaela Götz ,  Shriraghav Kaushik

IPC分类号： G06F17/30 ,  G06N99/00

CPC分类号： G06F17/30507 ,  G06N99/005

摘要： An active learning record matching system and method for producing a record matching package that is used to identify pairs of duplicate records. Embodiments of the system and method allow a precision threshold to be specified and then generate a learned record matching package having precision greater than this threshold and a recall close to the best possible recall. Embodiments of the system and method use a blocking technique to restrict the space of record matching packages considered and scale to large inputs. The learning method considers several record matching packages, estimates the precision and recall of the packages, and identifies the package with maximum recall having precision greater than equal to the given precision threshold. A human domain expert labels a sample of record pairs in the output of the package as matches or non-matches and this labeling is used to estimate the precision of the package.

摘要翻译： 用于产生用于识别重复记录对的记录匹配包的主动学习记录匹配系统和方法。 系统和方法的实施例允许指定精度阈值，然后产生具有大于该阈值的精度的学习记录匹配包以及接近最佳可能召回的召回。 系统和方法的实施例使用阻塞技术来限制所考虑的记录匹配包的空间并将其缩放到大的输入。 学习方法考虑了几个记录匹配包，估计包的精度和调用，并且识别具有大于等于给定精度阈值的精度的最大召回的包。 人类领域专家将包的输出中的记录对的样本标记为匹配或不匹配，并且该标签用于估计包的精度。

公开(公告)号：US20120259802A1

公开(公告)日：2012-10-11

申请号：US13084527

申请日：2011-04-11

申请人： Arvind Arasu ,  Michaela Götz ,  Shriraghav Kaushik

发明人： Arvind Arasu ,  Michaela Götz ,  Shriraghav Kaushik

IPC分类号： G06F15/18

CPC分类号： G06F17/30507 ,  G06N99/005

摘要： An active learning record matching system and method for producing a record matching package that is used to identify pairs of duplicate records. Embodiments of the system and method allow a precision threshold to be specified and then generate a learned record matching package having precision greater than this threshold and a recall close to the best possible recall. Embodiments of the system and method use a blocking technique to restrict the space of record matching packages considered and scale to large inputs. The learning method considers several record matching packages, estimates the precision and recall of the packages, and identifies the package with maximum recall having precision greater than equal to the given precision threshold. A human domain expert labels a sample of record pairs in the output of the package as matches or non-matches and this labeling is used to estimate the precision of the package.

摘要翻译： 用于产生用于识别重复记录对的记录匹配包的主动学习记录匹配系统和方法。 系统和方法的实施例允许指定精度阈值，然后产生具有大于该阈值的精度的学习记录匹配包以及接近最佳可能召回的召回。 系统和方法的实施例使用阻塞技术来限制所考虑的记录匹配包的空间并将其缩放到大的输入。 学习方法考虑了几个记录匹配包，估计包的精度和调用，并且识别具有大于等于给定精度阈值的精度的最大召回的包。 人类领域专家将包的输出中的记录对的样本标记为匹配或不匹配，并且该标签用于估计包的精度。

公开(公告)号：US20090210418A1

公开(公告)日：2009-08-20

申请号：US12031715

申请日：2008-02-15

申请人： Arvind Arasu ,  Surajit Chaudhuri ,  Shriraghav Kaushik

发明人： Arvind Arasu ,  Surajit Chaudhuri ,  Shriraghav Kaushik

IPC分类号： G06F17/30

CPC分类号： G06F17/30569 ,  G06F17/30675 ,  G06F17/30985

摘要： A transformation-based record matching technique. The technique provides a flexible way to account for synonyms and more general forms of string equivalences when performing record matching by taking as explicit input user-defined transformation rules (such as, for example, the fact that “Robert” and “Bob” that are synonymous). The input string and user-defined transformation rules are used to generate a larger set of strings which are used when performing record matching. Both the input string and data elements in a database can be transformed using the user-defined transformation rules in order to generate a larger set of potential record matches. These potential record matches can then be subjected to a threshold test in order to determine one or more best matches. Additionally, signature-based similarity functions are used to improve the computational efficiency of the technique.

摘要翻译： 基于变换的记录匹配技术。 当通过采用显式输入用户定义的转换规则（例如，“Robert”和“Bob”）这样的事实来执行记录匹配时，该技术提供了一种灵活的方式来解释同义词和更一般的字符串等同形式 同义词）。 输入字符串和用户定义的转换规则用于生成在执行记录匹配时使用的较大的一组字符串。 可以使用用户定义的变换规则来转换数据库中的输入字符串和数据元素，以便生成更大的潜在记录匹配集合。 然后可以对这些潜在的记录匹配进行阈值测试，以确定一个或多个最佳匹配。 另外，使用基于签名的相似度函数来提高该技术的计算效率。

公开(公告)号：US20090083238A1

公开(公告)日：2009-03-26

申请号：US11859046

申请日：2007-09-21

申请人： Surajit Chaudhuri ,  Shriraghav Kaushik ,  Abhijit Pol ,  Ravishankar Ramamurthy

发明人： Surajit Chaudhuri ,  Shriraghav Kaushik ,  Abhijit Pol ,  Ravishankar Ramamurthy

IPC分类号： G06F17/30

CPC分类号： G06F16/24561

摘要： Stop-and-restart query execution that partially leverages the work already performed during the initial execution of the query to reduce the execution time during a restart. The technique selectively saves information from a previous execution of the query so that the overhead associated with restarting the query execution can be bounded. Despite saving only limited information, the disclosed technique substantially reduces the running time of the restarted query. The stop-and-restart query execution technique is constrained to save and reuse only a bounded number of records (intermediate records or output records) thereby releasing all other resources, rather than some of the resources. The technique chooses a subset of the records to save that were found during normal execution and then skipping the corresponding records when performing a scan during restart to prevent the duplication of execution. A skip-scan operator is employed to facilitate the disclosed restart technique.

摘要翻译： 停止和重新启动的查询执行，部分利用在初始执行查询期间已经执行的工作，以减少重新启动期间的执行时间。 该技术选择性地保存来自查询的先前执行的信息，使得与重新启动查询执行相关联的开销可以被界定。 尽管仅节省有限的信息，但是所公开的技术大大减少了重新启动的查询的运行时间。 停止和重启查询执行技术被限制为只保存和重用有限数量的记录（中间记录或输出记录），从而释放所有其他资源，而不是一些资源。 该技术选择在正常执行期间发现的记录的子集，然后在重新启动期间执行扫描时跳过相应的记录，以防止重复执行。 采用跳过扫描运算符来促进公开的重启技术。

公开(公告)号：US20090006392A1

公开(公告)日：2009-01-01

申请号：US11769050

申请日：2007-06-27

申请人： Zhimin Chen ,  Venkatesh Ganti ,  Gunjan Jha ,  Shriraghav Kaushik ,  Vivek Narasayya

发明人： Zhimin Chen ,  Venkatesh Ganti ,  Gunjan Jha ,  Shriraghav Kaushik ,  Vivek Narasayya

IPC分类号： G06F7/06 ,  G06F17/30

CPC分类号： G06F17/30536

摘要： Architecture that provides a data profile computation technique which employs key profile computation and data pattern profile computation. Key profile computation in a data table includes both exact keys as well as approximate keys, and is based on key strengths. A key strength of 100% is an exact key, and any other percentage in an approximate key. The key strength is estimated based on the number of table rows that have duplicated attribute values. Only column sets that exceed a threshold value are returned. Pattern profiling identifies a small set of regular expression patterns which best describe the patterns within a given set of attribute values. Pattern profiling includes three phases: a first phases for determining token regular expressions, a second phase for determining candidate regular expressions, and a third phase for identifying the best regular expressions of the candidates that match the attribute values.

摘要翻译： 提供采用关键轮廓计算和数据模式轮廓计算的数据轮廓计算技术的架构。 数据表中的关键轮廓计算包括精密键和近似键，并且基于关键优点。 100％的关键优势是一个确切的关​​键，其中一个关键的任何其他百分比。 基于具有重复的属性值的表行的数量来估计关键强度。 只返回超过阈值的列集。 模式分析标识一组最佳描述一组给定属性值中的模式的正则表达式模式。 模式分析包括三个阶段：用于确定令牌正则表达式的第一阶段，用于确定候选正则表达式的第二阶段，以及用于识别与属性值匹配的候选的最佳正则表达式的第三阶段。

公开(公告)号：US20130132352A1

公开(公告)日：2013-05-23

申请号：US13303154

申请日：2011-11-23

申请人： Shriraghav Kaushik ,  Ravishankar Ramamurthy ,  Yupeng Fu

发明人： Shriraghav Kaushik ,  Ravishankar Ramamurthy ,  Yupeng Fu

IPC分类号： G06F17/30 ,  G06F7/00

CPC分类号： G06F16/2453 ,  G06F16/211

摘要： The present application provides for techniques for implementing data auditing embodiments that determine whether a query into a database is or has referenced forbidden data within the database. Various techniques are given for efficiently finding all tuples in a database referenced by a given query. A set of sensitive data is determined within a database and the set of sensitive data is employed to define a forbidden view within the database. Data within the database may be annotated to provide efficient identification of data access by query. Incoming queries may be analyzed and modified to propagate annotations for analyzing what data is or was accessed.

摘要翻译： 本申请提供了用于实现数据审计实施例的技术，其确定数据库中的查询是否是数据库内的或已经引用了数据库中的禁止数据。 给出了各种技术，用于有效地查找给定查询引用的数据库中的所有元组。 在数据库中确定一组敏感数据，并采用敏感数据集来定义数据库中的禁止视图。 可以注释数据库内的数据，以便通过查询提供对数据访问的有效识别。 可以分析和修改传入查询以传播用于分析数据被访问或被访问的注释。

公开(公告)号：US08204866B2

公开(公告)日：2012-06-19

申请号：US11804400

申请日：2007-05-18

申请人： Surajit Chaudhuri ,  Venkatesh Ganti ,  Shriraghav Kaushik ,  Anish Das Sarma

发明人： Surajit Chaudhuri ,  Venkatesh Ganti ,  Shriraghav Kaushik ,  Anish Das Sarma

IPC分类号： G06F17/30

CPC分类号： G06F17/30489

摘要： A deduplication algorithm that provides improved accuracy in data deduplication by using aggregate and/or groupwise constraints. Deduplication is accomplished using only as many of these constraints that are satisfied rather than be imposed inflexibly as hard constraints. Additionally, textual similarity between tuples is leveraged to restrict the search space. The algorithm begins with a coarse initial partition of data records and continues by raising the similarity threshold until the threshold splits a given partition. This sequence of splits defines a rich space of alternatives. Over this space, an algorithm finds a partition of the input that maximizes constraint satisfaction. In the context of groupwise aggregation constraints for deduplication all SQL (structured query language) aggregates are allowed, including summation.

摘要翻译： 重复数据删除算法，通过使用聚合和/或分组约束来提高重复数据删除的精度。 重复数据删除使用只有这些约束满足的约束才能实现，而不是将其作为硬约束条件强制强加。 此外，利用元组之间的文本相似性来限制搜索空间。 该算法以数据记录的粗略初始分区开始，并通过提高相似性阈值继续，直到阈值分裂给定分区。 这个拆分序列定义了丰富的替代空间。 在这个空间上，一个算法找到了一个最大化约束满足度的输入分区。 在重复数据消除的分组聚合约束的上下文中，允许所有SQL（结构化查询语言）聚合，包括求和。

公开(公告)号：US08046339B2

公开(公告)日：2011-10-25

申请号：US11758202

申请日：2007-06-05

申请人： Surajit Chaudhuri ,  Bee Chung Chen ,  Venkatesh Ganti ,  Shriraghav Kaushik

发明人： Surajit Chaudhuri ,  Bee Chung Chen ,  Venkatesh Ganti ,  Shriraghav Kaushik

IPC分类号： G06F17/30

CPC分类号： G06F17/30533 ,  G06F17/30495

摘要： Example-driven creation of record matching queries. The disclosed architecture employs techniques that exploit the availability of positive (or matching) and negative (non-matching) examples to search through this space and suggest an initial record matching query. The record matching task is modeled as that of designing an operator tree obtained by composing a few primitive operators. This ensures that record matching programs be executable efficiently and scalably over large input relations. The architecture joins records across multiple (e.g., two) relations (e.g., R and S). The architecture exploits the monotonicity property of similarity functions for record matching in the relations, in that, any pair of matching records have a higher similarity value than non-matching record pairs on at least one similarity function.

摘要翻译： 示例驱动创建记录匹配查询。 所公开的架构采用利用正（或匹配）和否定（不匹配）示例的可用性来搜索该空间并提出初始记录匹配查询的技术。 记录匹配任务被建模为设计通过组合几个原始算子获得的运算符树的记录匹配任务。 这确保了记录匹配程序可以在大的输入关系上有效和可扩展地执行。 该架构通过多个（例如，两个）关系（例如，R和S）连接记录。 该架构利用了关系中记录匹配的相似度函数的单调性，因为任何一对匹配记录具有比至少一个相似度函数上的非匹配记录对更高的相似度值。

公开(公告)号：US07610283B2

公开(公告)日：2009-10-27

申请号：US11761425

申请日：2007-06-12

申请人： Arvind Arasu ,  Venkatesh Ganti ,  Shriraghav Kaushik

发明人： Arvind Arasu ,  Venkatesh Ganti ,  Shriraghav Kaushik

IPC分类号： G06F17/30 ,  G06F7/06 ,  G06F7/08 ,  G06F7/10

CPC分类号： G06F17/30312 ,  Y10S707/99931 ,  Y10S707/99932 ,  Y10S707/99933 ,  Y10S707/99935 ,  Y10S707/99937

摘要： Input set indexing for set-similarity lookups. The architecture provides input to an indexing process that enables more efficient lookups for large data sets (e.g., disk-based) without requiring a full scan of the input. A new index structure is provided, the output of which is exact, rather than approximate. The similarity of two sets is specified using a similarity function that maps two sets to a numeric value that represents similarity of the two sets. Threshold-based lookups are addressed where two sets are considered similar if the numeric similarity score is above a threshold. The structure efficiently identifies all input sets within a distance k (e.g., a hamming distance) of the query set. Additional information in the form of frequency of elements (the number of input sets in which an element occurs) is used to improve index performance.

摘要翻译： 用于集合相似性查找的输入集索引。 该体系结构为索引过程提供输入，可以对大数据集（例如，基于磁盘）进行更有效的查找，而无需对输入进行全面扫描。 提供了一个新的索引结构，其输出是精确的，而不是近似的。 使用将两组映射到表示两组相似度的数值的相似度函数来指定两组的相似度。 如果数字相似性分数高于阈值，则基于阈值的查找被解决为其中两个集合被认为是相似的。 该结构有效地识别查询集合的距离k（例如，汉明距离）内的所有输入集合。 使用元素频率（元素发生的输入集合的数量）的形式的附加信息用于提高索引性能。

公开(公告)号：US20080288482A1

公开(公告)日：2008-11-20

申请号：US11804400

申请日：2007-05-18

申请人： Surajit Chaudhuri ,  Venkatesh Ganti ,  Shriraghav Kaushik

发明人： Surajit Chaudhuri ,  Venkatesh Ganti ,  Shriraghav Kaushik

IPC分类号： G06F17/30

CPC分类号： G06F17/30489

摘要： A deduplication algorithm that provides improved accuracy in data deduplication by using aggregate and/or groupwise constraints. Deduplication is accomplished using only as many of these constraints that are satisfied rather than be imposed inflexibly as hard constraints. Additionally, textual similarity between tuples is leveraged to restrict the search space. The algorithm begins with a coarse initial partition of data records and continues by raising the similarity threshold until the threshold splits a given partition. This sequence of splits defines a rich space of alternatives. Over this space, an algorithm finds a partition of the input that maximizes constraint satisfaction. In the context of groupwise aggregation constraints for deduplication all SQL (structured query language) aggregates are allowed, including summation.

摘要翻译： 重复数据删除算法，通过使用聚合和/或分组约束来提高重复数据删除的精度。 重复数据删除使用只有这些约束满足的约束才能实现，而不是将其作为硬约束条件强制强加。 此外，利用元组之间的文本相似性来限制搜索空间。 该算法以数据记录的粗略初始分区开始，并通过提高相似性阈值继续，直到阈值分裂给定分区。 这个拆分序列定义了丰富的替代空间。 在这个空间上，一个算法找到了一个最大化约束满足度的输入分区。 在重复数据消除的分组聚合约束的上下文中，允许所有SQL（结构化查询语言）聚合，包括求和。