公开(公告)号：US07249121B1

公开(公告)日：2007-07-24

申请号：US09729240

申请日：2000-12-05

申请人： Krishna Bharat ,  Sanjay Ghemawat ,  Urs Hoelzle

发明人： Krishna Bharat ,  Sanjay Ghemawat ,  Urs Hoelzle

IPC分类号： G06F17/30 ,  G06F15/16

CPC分类号： G06F17/30867 ,  G06F17/30663 ,  Y10S707/99931 ,  Y10S707/99933

摘要： A search engine for searching a corpus improves the relevancy of the results by classifying multiple terms in a search query as a single semantic unit. A semantic unit locator of the search engine generates a subset of documents that are generally relevant to the query based on the individual terms within the query. Combinations of search terms that define potential semantic units from the query are then evaluated against the subset of documents to determine which combinations of search terms should be classified as a semantic unit. The resultant semantic units are used to refine the results of the search.

摘要翻译： 用于搜索语料库的搜索引擎通过将搜索查询中的多个项目分类为单个语义单元来提高结果的相关性。 搜索引擎的语义单元定位器基于查询中的各个术语生成通常与查询相关的文档的子集。 然后根据文档子集来评估从查询定义潜在语义单元的搜索项的组合，以确定搜索词的哪些组合应该被分类为语义单元。 所得到的语义单位用于细化搜索结果。

公开(公告)号：US08719262B1

公开(公告)日：2014-05-06

申请号：US13616094

申请日：2012-09-14

申请人： Krishna Bharat ,  Sanjay Ghemawat ,  Urs Hoelzle

发明人： Krishna Bharat ,  Sanjay Ghemawat ,  Urs Hoelzle

IPC分类号： G06F17/30

CPC分类号： G06F17/30867 ,  G06F17/30663 ,  Y10S707/99931 ,  Y10S707/99933

摘要： A search engine for searching a corpus improves the relevancy of the results by classifying multiple terms in a search query as a single semantic unit. A semantic unit locator of the search engine generates a subset of documents that are generally relevant to the query based on the individual terms within the query. Combinations of search terms that define potential semantic units from the query are then evaluated against the subset of documents to determine which combinations of search terms should be classified as a semantic unit. The resultant semantic units are used to refine the results of the search.

摘要翻译： 用于搜索语料库的搜索引擎通过将搜索查询中的多个项目分类为单个语义单元来提高结果的相关性。 搜索引擎的语义单元定位器基于查询中的各个术语生成通常与查询相关的文档的子集。 然后根据文档子集来评估从查询定义潜在语义单元的搜索项的组合，以确定搜索词的哪些组合应该被分类为语义单元。 所得到的语义单位用于细化搜索结果。

公开(公告)号：US08321410B1

公开(公告)日：2012-11-27

申请号：US11764566

申请日：2007-06-18

申请人： Krishna Bharat ,  Sanjay Ghemawat ,  Urs Hoelzle

发明人： Krishna Bharat ,  Sanjay Ghemawat ,  Urs Hoelzle

IPC分类号： G06F17/30

CPC分类号： G06F17/30867 ,  G06F17/30663 ,  Y10S707/99931 ,  Y10S707/99933

摘要： A search engine for searching a corpus improves the relevancy of the results by classifying multiple terms in a search query as a single semantic unit. A semantic unit locator of the search engine generates a subset of documents that are generally relevant to the query based on the individual terms within the query. Combinations of search terms that define potential semantic units from the query are then evaluated against the subset of documents to determine which combinations of search terms should be classified as a semantic unit. The resultant semantic units are used to refine the results of the search.

摘要翻译： 用于搜索语料库的搜索引擎通过将搜索查询中的多个项目分类为单个语义单元来提高结果的相关性。 搜索引擎的语义单元定位器基于查询中的各个术语生成通常与查询相关的文档的子集。 然后根据文档子集来评估从查询定义潜在语义单元的搜索项的组合，以确定搜索词的哪些组合应该被分类为语义单元。 所得到的语义单位用于细化搜索结果。

公开(公告)号：US09405808B2

公开(公告)日：2016-08-02

申请号：US13407632

申请日：2012-02-28

申请人： Robert C. Pike ,  Sean Quinlan ,  Sean M. Dorward ,  Jeffrey Dean ,  Sanjay Ghemawat

发明人： Robert C. Pike ,  Sean Quinlan ,  Sean M. Dorward ,  Jeffrey Dean ,  Sanjay Ghemawat

IPC分类号： G06F17/30 ,  G06F11/14

CPC分类号： G06F17/30501 ,  G06F11/1482 ,  G06F17/30545 ,  G06F17/30598 ,  Y10S707/99933 ,  Y10S707/99937

摘要： A method and system for analyzing data records includes allocating groups of records to respective processes of a first plurality of processes executing in parallel. In each respective process of the first plurality of processes, for each record in the group of records allocated to the respective process, a query is applied to the record so as to produce zero or more values. Zero or more emit operators are applied to each of the zero or more produced values so as to add corresponding information to an intermediate data structure. Information from a plurality of the intermediate data structures is aggregated to produce output data.

公开(公告)号：US08868559B2

公开(公告)日：2014-10-21

申请号：US13599707

申请日：2012-08-30

申请人： Daniel Dulitz ,  Alexandre A. Verstak ,  Sanjay Ghemawat ,  Jeffrey A. Dean

发明人： Daniel Dulitz ,  Alexandre A. Verstak ,  Sanjay Ghemawat ,  Jeffrey A. Dean

IPC分类号： G06F7/00 ,  G06F17/30

CPC分类号： G06F17/30867 ,  G06F17/3053 ,  G06F17/3071 ,  G06F17/30864 ,  Y10S707/99931 ,  Y10S707/99932 ,  Y10S707/99935 ,  Y10S707/99954

摘要： Systems and methods for indexing a representative document from a set of duplicate documents are disclosed. Disclosed systems and methods comprise selecting a first document in a plurality of documents on the basis that the first document is associated with a query independent score. Each respective document in the plurality of documents has a fingerprint that indicates that the respective document has substantially identical content to every other document in the plurality of documents. Disclosed systems and methods further comprise indexing, in accordance with the query independent score, the first document thereby producing an indexed first document. With respect to the plurality of documents, only the indexed first document is included in a document index.

摘要翻译： 公开了从一组重复文件索引代表性文件的系统和方法。 公开的系统和方法包括在第一文档与查询独立分数相关联的基础上选择多个文档中的第一文档。 多个文档中的每个文档具有指示相应文档具有与多个文档中的每个其他文档基本相同的内容的指纹。 公开的系统和方法还包括根据查询独立分数索引第一文档，从而产生索引的第一文档。 对于多个文档，仅索引的第一文档被包括在文档索引中。

公开(公告)号：US08504518B1

公开(公告)日：2013-08-06

申请号：US13274043

申请日：2011-10-14

申请人： Sanjay Ghemawat ,  Howard Gobioff ,  Shun-Tak Leung

发明人： Sanjay Ghemawat ,  Howard Gobioff ,  Shun-Tak Leung

IPC分类号： G06F17/30

CPC分类号： H04L67/1095 ,  G06F17/30174 ,  G06F17/30215

摘要： A system that facilitates the distribution and redistribution of chunks of data among multiple servers, may identify servers to store replicas of the chunks based on at least one of utilization, prior data distribution, and failure correlation properties, and place the replicas at the identified servers. The system may monitor total numbers of replicas available in the system, identify chunks that have a total number of replicas below one or more thresholds, assign priorities to the identified chunks, and re-replicate the identified chunks based on the assigned priorities. The system may monitor utilization of the servers, select one or more of the replicas to redistribute based on the utilization of the servers, select one or more of the servers to which to move the one or more replicas, and move the one or more replicas to the selected one or more servers.

摘要翻译： 有助于在多个服务器之间分发和重新分发数据块的系统可以基于利用，先前的数据分布和故障相关属性中的至少一个来识别存储块的副本的服务器，并将副本放置在所识别的服务器 。 该系统可以监视系统中可用的副本的总数，将具有低于一个或多个阈值的副本总数的块识别为所识别的块分配优先级，并且基于所分配的优先级重新复制所识别的块。 系统可以监视服务器的利用率，基于服务器的利用率选择一个或多个副本重新分发，选择要移动一个或多个副本的一个或多个服务器，并移动一个或多个副本 到所选择的一个或多个服务器。

公开(公告)号：US08495483B1

公开(公告)日：2013-07-23

申请号：US10386110

申请日：2003-03-12

申请人： Jeffrey A. Dean ,  Martin Farach-Colton ,  Sanjay Ghemawat ,  Benedict Gomes ,  Georges R. Hank

发明人： Jeffrey A. Dean ,  Martin Farach-Colton ,  Sanjay Ghemawat ,  Benedict Gomes ,  Georges R. Hank

IPC分类号： G06F17/00 ,  G06F17/30

CPC分类号： G06F17/30864

摘要： Web quotes are gathered from web pages that link to a web page of interest. The web quote may include text from the paragraphs that contain the hypertext links to the page of interest as well as text from other portions of the linked web page, such as text from a nearby header. The obtained web quotes may be ranked based on quality or relevance and may then be incorporated into a search engine's document index or into summary information returned to users in response to a search query.

摘要翻译： 网络引用从链接到感兴趣的网页的网页收集。 网络报价可以包括来自包含到感兴趣页面的超文本链接的段落的文本以及链接网页的其他部分的文本，例如来自附近标题的文本。 获得的网络报价可以基于质量或相关性来排序，然后可以被合并到搜索引擎的文档索引中或者被合并到响应于搜索查询返回给用户的摘要信息中。

公开(公告)号：US20120323896A1

公开(公告)日：2012-12-20

申请号：US13599707

申请日：2012-08-30

申请人： Daniel Dulitz ,  Alexandre A. Verstak ,  Sanjay Ghemawat ,  Jeffrey A. Dean

发明人： Daniel Dulitz ,  Alexandre A. Verstak ,  Sanjay Ghemawat ,  Jeffrey A. Dean

IPC分类号： G06F17/30

CPC分类号： G06F17/30867 ,  G06F17/3053 ,  G06F17/3071 ,  G06F17/30864 ,  Y10S707/99931 ,  Y10S707/99932 ,  Y10S707/99935 ,  Y10S707/99954

摘要： Systems and methods for indexing a representative document from a set of duplicate documents are disclosed. Disclosed systems and methods comprise selecting a first document in a plurality of documents on the basis that the first document is associated with a query independent score. Each respective document in the plurality of documents has a fingerprint that indicates that the respective document has substantially identical content to every other document in the plurality of documents. Disclosed systems and methods further comprise indexing, in accordance with the query independent score, the first document thereby producing an indexed first document. With respect to the plurality of documents, only the indexed first document is included in a document index.

摘要翻译： 公开了从一组重复文件索引代表性文件的系统和方法。 公开的系统和方法包括在第一文档与查询独立分数相关联的基础上选择多个文档中的第一文档。 多个文档中的每个文档具有指示相应文档具有与多个文档中的每个其他文档基本相同的内容的指纹。 公开的系统和方法还包括根据查询独立分数索引第一文档，从而产生索引的第一文档。 对于多个文档，仅索引的第一文档被包括在文档索引中。

公开(公告)号：US08260781B2

公开(公告)日：2012-09-04

申请号：US13186414

申请日：2011-07-19

申请人： Daniel Dulitz ,  Alexandre A. Verstak ,  Sanjay Ghemawat ,  Jeffrey A. Dean

发明人： Daniel Dulitz ,  Alexandre A. Verstak ,  Sanjay Ghemawat ,  Jeffrey A. Dean

IPC分类号： G06F7/00 ,  G06F17/30

CPC分类号： G06F17/30867 ,  G06F17/3053 ,  G06F17/3071 ,  G06F17/30864 ,  Y10S707/99931 ,  Y10S707/99932 ,  Y10S707/99935 ,  Y10S707/99954

摘要： Duplicate documents are detected in a web crawler system. Upon receiving a newly crawled document, a set of documents, if any, sharing the same content as the newly crawled document is identified. Information identifying the newly crawled document and the selected set of documents is merged into information identifying a new set of documents. Duplicate documents are included and excluded from the new set of documents based on a query independent metric for each such document. A single representative document for the new set of documents is identified in accordance with a set of predefined conditions.

摘要翻译： 在网页抓取工具系统中检测到重复的文档。 在接收到新爬取的文档时，识别与新爬取的文档共享相同内容的一组文档（如果有的话）。 识别新爬取的文档和所选择的一组文档的信息被合并到识别新的一组文档的信息中。 基于每个此类文档的查询独立指标，将重复的文档包含在新文档集中并从其中排除。 根据一组预定义的条件识别新的文档集合的单个代表性文档。

公开(公告)号：US20120215787A1

公开(公告)日：2012-08-23

申请号：US13407632

申请日：2012-02-28

申请人： Robert C. Pike ,  Sean Quinlan ,  Sean M. Dorward ,  Jeffrey Dean ,  Sanjay Ghemawat

发明人： Robert C. Pike ,  Sean Quinlan ,  Sean M. Dorward ,  Jeffrey Dean ,  Sanjay Ghemawat

IPC分类号： G06F17/30

CPC分类号： G06F17/30501 ,  G06F11/1482 ,  G06F17/30545 ,  G06F17/30598 ,  Y10S707/99933 ,  Y10S707/99937

摘要： A method and system for analyzing data records includes allocating groups of records to respective processes of a first plurality of processes executing in parallel. In each respective process of the first plurality of processes, for each record in the group of records allocated to the respective process, a query is applied to the record so as to produce zero or more values. Zero or more emit operators are applied to each of the zero or more produced values so as to add corresponding information to an intermediate data structure. Information from a plurality of the intermediate data structures is aggregated to produce output data.

摘要翻译： 用于分析数据记录的方法和系统包括：将记录组分配给并行执行的第一多个进程的各个进程。 在第一多个处理的每个相应处理中，对于分配给相应处理的记录组中的每个记录，将对该记录应用查询以产生零个或多个值。 将零个或更多个发射操作符应用于零或更多产生的值中的每一个，以便将相应的信息添加到中间数据结构。 来自多个中间数据结构的信息被聚合以产生输出数据。