摘要:
A distributed data reorganization system and method for mapping and reducing raw data containing a plurality of data records. Embodiments of the distributed data reorganization system and method operate in a general-purpose parallel execution environment that use an arbitrary communication directed acyclic graph. The vertices of the graph accept multiple data inputs and generate multiple data inputs, and may be of different types. Embodiments of the distributed data reorganization system and method include a plurality of distributed mappers that use a mapping criteria supplied by a developer to map the plurality of data records to data buckets. The mapped data record and data bucket identifications are input for a plurality of distributed reducers. Each distributed reducer groups together data records having the same data bucket identification and then uses a merge logic supplied by the developer to reduce the grouped data records to obtain reorganized data.
摘要:
Embodiments for caching and accessing Directed Acyclic Graph (DAG) data to and from a computing device of a DAG distributed execution engine during the processing of an iterative algorithm. In accordance with one embodiment, a method includes processing a first subgraph of the plurality of subgraphs from the distributed storage system in the computing device. The first subgraph being processed with associated input values in the computing device to generate first output values in an iteration. The method further includes storing a second subgraph in a cache of the device. The second subgraph being a duplicate of the first subgraph. Moreover, the method also includes processing the second subgraph with the first output values to generate second output values if the device is to process the first subgraph in each of one or more subsequent iterations.
摘要:
A task guidance tool that displays instructional steps and associated advertisements may facilitate the accomplishment of a task by users who are otherwise unfamiliar with the task. The task guidance tool may be developed from input data mined from various sources. The task guidance tool may display a series of step pages in which each step page include instructions for accomplishing a corresponding step of the task. Further, one or more step pages of the task guidance tool may be provided with selected advertisements that are displayed with the step instructions.
摘要:
Techniques for determining user information needs and selecting data based on user information needs are described herein. The present disclosure describes extracting topics of interests to users from multiple sources including search log data and social network website, and assigns a budget to each topic to stipulate the quota of data to be selected for each topic. The present disclosure also describes calculating similarities between gathered data and the topics, and selecting top related data with each topic subject to limit of the budget. A search engine may use the techniques described here to select data for its index.
摘要:
Systems, methods, and devices for sorting and processing various types of graph data are described herein. Partitioning graph data into master data and associated slave data allows for sorting of the graph data by sorting the master data. In another embodiment, promoting a data bucket having a first data bucket size to a data bucket having a second data bucket size greater than the first data bucket size upon reaching a memory limit allows for the reduction of temporary files output by the data bucket.
摘要:
Some implementations provide techniques for selecting web pages for inclusion in an index. For example, some implementations apply regularization to select a subset of the crawled web pages for indexing based on link relationships between the crawled web pages, features extracted from the crawled web pages, and user behavior information determined for at least some of the crawled web pages. Further, in some implementations, the user behavior information may be used to sort a training set of crawled web pages into a plurality of labeled groups. The labeled groups may be represented in a directed graph that indicates relative priorities for being selected for indexing.
摘要:
Importance ranking of web pages is performed by defining a graph-based regularization term based on document features, edge features, and a web graph of a plurality of web pages, and deriving a loss term based on human feedback data. The graph-based regularization term and the loss term are combined to obtain a global objective function. The global objective function is optimized to obtain parameters for the document features and edge features and to produce static rank scores for the plurality of web pages. Further, the plurality of web pages is ordered based on the static rank scores.
摘要:
This application describes a system and method for estimating user intent towards categories of content. The estimation of user intent may be based at least in part on a score for prior user actions and a decay function that is applied to that score to provide an estimate of current user intent. The estimate represents current user intent for time periods in which user actions towards a category of content are negligible or non-existent.
摘要:
Method for determining a webpage importance, including receiving web browsing behavior data of one or more users; creating a model of the web browsing behavior data; calculating a stationary probability distribution of the model; and correlating the stationary probability distribution to the webpage importance.
摘要:
A calculate importance system calculates the global importance of a web page based on a “mean hitting time.” Hitting time of a target web page is a measure of the minimum number of transitions needed to land on the target web page. Mean hitting time of a target web page is an average number of such transitions for all possible starting web pages. The calculate importance system calculates a global importance score for a web page based on the reciprocal of a mean hitting time. A search engine may rank web pages of a search result based on a combination of relevance of the web pages to the search request and global importance of the web pages based on a global hitting time.