-
1.发明授权Representative document selection for sets of duplicate documents in a web crawler system 有权Web爬网系统中的重复文件集的代表性文档选择公开(公告)号：US07984054B2
公开(公告)日：2011-07-19
申请号：US12628989
申请日：2009-12-01
CPC分类号： G06F17/30867 , G06F17/3053 , G06F17/3071 , G06F17/30864 , Y10S707/99931 , Y10S707/99932 , Y10S707/99935 , Y10S707/99954
摘要： Duplicate documents are detected in a web crawler system. Upon receiving a newly crawled document, a set of documents, if any, sharing the same content as the newly crawled document is identified. Information identifying the newly crawled document and the selected set of documents is merged into information identifying a new set of documents. Duplicate documents are included and excluded from the new set of documents based on a query independent metric for each such document. A single representative document for the new set of documents is identified in accordance with a set of predefined conditions.
摘要翻译： 在网页抓取工具系统中检测到重复的文档。 在接收到新爬取的文档时，识别与新爬取的文档共享相同内容的一组文档（如果有的话）。 识别新爬取的文档和所选择的一组文档的信息被合并到识别新的一组文档的信息中。 基于每个此类文档的查询独立指标，将重复的文档包含在新文档集中并排除在外。 根据一组预定义的条件识别新的文档集合的单个代表性文档。
-
公开(公告)号：US08868559B2
公开(公告)日：2014-10-21
申请号：US13599707
申请日：2012-08-30
CPC分类号： G06F17/30867 , G06F17/3053 , G06F17/3071 , G06F17/30864 , Y10S707/99931 , Y10S707/99932 , Y10S707/99935 , Y10S707/99954
摘要： Systems and methods for indexing a representative document from a set of duplicate documents are disclosed. Disclosed systems and methods comprise selecting a first document in a plurality of documents on the basis that the first document is associated with a query independent score. Each respective document in the plurality of documents has a fingerprint that indicates that the respective document has substantially identical content to every other document in the plurality of documents. Disclosed systems and methods further comprise indexing, in accordance with the query independent score, the first document thereby producing an indexed first document. With respect to the plurality of documents, only the indexed first document is included in a document index.
摘要翻译： 公开了从一组重复文件索引代表性文件的系统和方法。 公开的系统和方法包括在第一文档与查询独立分数相关联的基础上选择多个文档中的第一文档。 多个文档中的每个文档具有指示相应文档具有与多个文档中的每个其他文档基本相同的内容的指纹。 公开的系统和方法还包括根据查询独立分数索引第一文档，从而产生索引的第一文档。 对于多个文档，仅索引的第一文档被包括在文档索引中。
-
公开(公告)号：US20120323896A1
公开(公告)日：2012-12-20
申请号：US13599707
申请日：2012-08-30
IPC分类号： G06F17/30
CPC分类号： G06F17/30867 , G06F17/3053 , G06F17/3071 , G06F17/30864 , Y10S707/99931 , Y10S707/99932 , Y10S707/99935 , Y10S707/99954
摘要： Systems and methods for indexing a representative document from a set of duplicate documents are disclosed. Disclosed systems and methods comprise selecting a first document in a plurality of documents on the basis that the first document is associated with a query independent score. Each respective document in the plurality of documents has a fingerprint that indicates that the respective document has substantially identical content to every other document in the plurality of documents. Disclosed systems and methods further comprise indexing, in accordance with the query independent score, the first document thereby producing an indexed first document. With respect to the plurality of documents, only the indexed first document is included in a document index.
摘要翻译： 公开了从一组重复文件索引代表性文件的系统和方法。 公开的系统和方法包括在第一文档与查询独立分数相关联的基础上选择多个文档中的第一文档。 多个文档中的每个文档具有指示相应文档具有与多个文档中的每个其他文档基本相同的内容的指纹。 公开的系统和方法还包括根据查询独立分数索引第一文档，从而产生索引的第一文档。 对于多个文档，仅索引的第一文档被包括在文档索引中。
-
4.发明授权Representative document selection for sets of duplicate documents in a web crawler system 有权Web爬网系统中的重复文件集的代表性文档选择公开(公告)号：US08260781B2
公开(公告)日：2012-09-04
申请号：US13186414
申请日：2011-07-19
CPC分类号： G06F17/30867 , G06F17/3053 , G06F17/3071 , G06F17/30864 , Y10S707/99931 , Y10S707/99932 , Y10S707/99935 , Y10S707/99954
摘要： Duplicate documents are detected in a web crawler system. Upon receiving a newly crawled document, a set of documents, if any, sharing the same content as the newly crawled document is identified. Information identifying the newly crawled document and the selected set of documents is merged into information identifying a new set of documents. Duplicate documents are included and excluded from the new set of documents based on a query independent metric for each such document. A single representative document for the new set of documents is identified in accordance with a set of predefined conditions.
摘要翻译： 在网页抓取工具系统中检测到重复的文档。 在接收到新爬取的文档时，识别与新爬取的文档共享相同内容的一组文档（如果有的话）。 识别新爬取的文档和所选择的一组文档的信息被合并到识别新的一组文档的信息中。 基于每个此类文档的查询独立指标，将重复的文档包含在新文档集中并从其中排除。 根据一组预定义的条件识别新的文档集合的单个代表性文档。
-
5.发明申请Representative Document Selection for Sets of Duplicate Documents in a Web Crawler System 有权Web爬虫系统中重复文档集的代表性文档选择公开(公告)号：US20110276561A1
公开(公告)日：2011-11-10
申请号：US13186414
申请日：2011-07-19
IPC分类号： G06F17/30
CPC分类号： G06F17/30867 , G06F17/3053 , G06F17/3071 , G06F17/30864 , Y10S707/99931 , Y10S707/99932 , Y10S707/99935 , Y10S707/99954
摘要： Duplicate documents are detected in a web crawler system. Upon receiving a newly crawled document, a set of documents, if any, sharing the same content as the newly crawled document is identified. Information identifying the newly crawled document and the selected set of documents is merged into information identifying a new set of documents. Duplicate documents are included and excluded from the new set of documents based on a query independent metric for each such document. A single representative document for the new set of documents is identified in accordance with a set of predefined conditions.
摘要翻译： 在网页抓取工具系统中检测到重复的文档。 在接收到新爬取的文档时，识别与新爬取的文档共享相同内容的一组文档（如果有的话）。 识别新爬取的文档和所选择的一组文档的信息被合并到识别新的一组文档的信息中。 基于每个此类文档的查询独立指标，将重复的文档包含在新文档集中并排除在外。 根据一组预定义的条件识别新的文档集合的单个代表性文档。
-
6.发明申请公开(公告)号：US20100076954A1
公开(公告)日：2010-03-25
申请号：US12628989
申请日：2009-12-01
IPC分类号： G06F17/30
CPC分类号： G06F17/30867 , G06F17/3053 , G06F17/3071 , G06F17/30864 , Y10S707/99931 , Y10S707/99932 , Y10S707/99935 , Y10S707/99954
摘要： Duplicate documents are detected in a web crawler system. Upon receiving a newly crawled document, a set of documents, if any, sharing the same content as the newly crawled document is identified. Information identifying the newly crawled document and the selected set of documents is merged into information identifying a new set of documents. Duplicate documents are included and excluded from the new set of documents based on a query independent metric for each such document. A single representative document for the new set of documents is identified in accordance with a set of predefined conditions.
摘要翻译： 在网页抓取工具系统中检测到重复的文档。 在接收到新爬取的文档时，识别与新爬取的文档共享相同内容的一组文档（如果有的话）。 识别新爬取的文档和所选择的一组文档的信息被合并到识别新的一组文档的信息中。 基于每个此类文档的查询独立指标，将重复的文档包含在新文档集中并从其中排除。 根据一组预定义的条件识别新的文档集合的单个代表性文档。
-
公开(公告)号：US07627613B1
公开(公告)日：2009-12-01
申请号：US10614111
申请日：2003-07-03
CPC分类号： G06F17/30867 , G06F17/3053 , G06F17/3071 , G06F17/30864 , Y10S707/99931 , Y10S707/99932 , Y10S707/99935 , Y10S707/99954
摘要： Duplicate documents are detected in a web crawler system. Upon receiving a newly crawled document, a set of documents, if any, sharing the same content as the newly crawled document is identified. Information identifying the newly crawled document and the selected set of documents is merged into information identifying a new set of documents. Duplicate documents are included and excluded from the new set of documents based on a query independent metric for each such document. A single representative document for the new set of documents is identified in accordance with a set of predefined conditions.
摘要翻译： 在网页抓取工具系统中检测到重复的文档。 在接收到新爬取的文档时，识别与新爬取的文档共享相同内容的一组文档（如果有的话）。 识别新爬取的文档和所选择的一组文档的信息被合并到识别新的一组文档的信息中。 基于每个此类文档的查询独立指标，将重复的文档包含在新文档集中并从其中排除。 根据一组预定义的条件识别新的文档集合的单个代表性文档。
-
8.发明授权Using text surrounding hypertext links when indexing and generating page summaries 有权在索引和生成页面摘要时使用超文本链接的文本公开(公告)号：US08495483B1
公开(公告)日：2013-07-23
申请号：US10386110
申请日：2003-03-12
CPC分类号： G06F17/30864
摘要： Web quotes are gathered from web pages that link to a web page of interest. The web quote may include text from the paragraphs that contain the hypertext links to the page of interest as well as text from other portions of the linked web page, such as text from a nearby header. The obtained web quotes may be ranked based on quality or relevance and may then be incorporated into a search engine's document index or into summary information returned to users in response to a search query.
摘要翻译： 网络引用从链接到感兴趣的网页的网页收集。 网络报价可以包括来自包含到感兴趣页面的超文本链接的段落的文本以及链接网页的其他部分的文本，例如来自附近标题的文本。 获得的网络报价可以基于质量或相关性来排序，然后可以被合并到搜索引擎的文档索引中或者被合并到响应于搜索查询返回给用户的摘要信息中。
-
公开(公告)号：US08244530B2
公开(公告)日：2012-08-14
申请号：US13249136
申请日：2011-09-29
IPC分类号： G10L15/06
CPC分类号： G06F17/3071
摘要： A set of documents may be stored and indexed as a compressed sequence of tokens. A set of documents are grouped into clusters. Sequences of tokens representing the clusters of documents are encoded to elide some repeating instances of tokens. A compressed sequence of tokens is generated from the compressed cluster sequences of tokens. Queries on the compressed sequence are performed by identifying cluster sequences within the compressed sequence that are likely to have documents that satisfy the query and then identifying, within these identified clusters, the documents that actually satisfies the query.
摘要翻译： 可以将一组文档存储并索引为压缩的令牌序列。 一组文档被分组成簇。 代表文档集群的令牌序列被编码，以清除令牌的一些重复实例。 从令牌的压缩簇序列生成令牌的压缩序列。 通过识别压缩序列中可能具有满足查询的文档，然后在这些标识的集群中识别实际满足查询的文档来执行对压缩序列的查询。
-
公开(公告)号：US07548928B1
公开(公告)日：2009-06-16
申请号：US11197922
申请日：2005-08-05
申请人： Jeffrey A. Dean , Sanjay Ghemawat
发明人： Jeffrey A. Dean , Sanjay Ghemawat
IPC分类号： G06F17/30
CPC分类号： G06F17/30985 , H03M7/3084 , Y10S707/99942 , Y10S707/99945
摘要： A method of compressing data in a table data structure begins by accessing a data set within the table data structure, the data set having associated therewith a range of rows of the table data structure. Data items in the data set are represented by key-value pairs. The method includes applying a first compression to the values of the key-value pairs in the data set to produce a first compressed output; applying a second compression, distinct from the first compression, to the keys of the key-value pairs in the data set to produce a second compressed output; and applying a third compression to the first compressed output and second compressed output to produce a first compressed output block, wherein the third compression is distinct from the first compression and second compression.
摘要翻译： 一种压缩表格数据结构中的数据的方法是通过访问表格数据结构内的数据集开始的，该数据集与表数据结构中的一行行相关联。 数据集中的数据项由键值对表示。 该方法包括对数据集中的键值对的值应用第一压缩以产生第一压缩输出; 将不同于第一压缩的第二压缩应用于数据集中的键值对的键以产生第二压缩输出; 以及将第三压缩应用于所述第一压缩输出和所述第二压缩输出以产生第一压缩输出块，其中所述第三压缩与所述第一压缩和所述第二压缩不同。
-
-
-
-
-
-
-
-
-
搜索结果
68 条记录 (耗时 0.029 秒)
