Abstract:
Systems and methods are provided for a personal entity modeling for computing devices. For example, a computing device comprises at least one processor and memory storing instructions that, when executed by the at least one processor, cause the mobile device to perform operations including identifying a personal entity in content generated for display on the mobile device, generating training examples for the personal entity from the content, and updating an embedding used to model the personal entity using the training examples. The embedding may be used to make predictions regarding the personal entity. For example, the operations may also include predicting an association between a first personal entity displayed on the computing device and a second entity based on the embedding, and providing a recommendation, to be displayed on the computing device, related to the second entity.
Abstract:
Methods, systems, and apparatus for receiving a query image and a user tap location, processing the received query image based on the user tap location, identifying one or more entities associated with the processed query image and in response to receiving (i) the query image, and (ii) the user tap location, providing information about the identified one or more of the entities.
Abstract:
Methods, systems, and apparatus for receiving a query image, receiving one or more entities that are associated with the query image, identifying, for one or more of the entities, one or more candidate search queries that are pre-associated with the one or more entities, generating a respective relevance score for each of the candidate search queries, selecting, as a representative search query for the query image, a particular candidate search query based at least on the generated respective relevance scores and providing the representative search query for output in response to receiving the query image.
Abstract:
Systems and methods are provided for suggesting actions for entities discovered in content on a mobile device. An example method can include running a mobile device emulator with a deep-link for a mobile application, determining a main entity for the deep link, mapping the main entity to the deep link, storing the mapping of the main entity to the deep link in a memory, and providing the mapping to a mobile device, the mapping enabling a user of the mobile device to select the deep link when the main entity is displayed on a screen of the mobile device. Another example method can include identifying at least one entity in content generated by a mobile application, identifying an action mapped to the at least one entity, the action representing a deep link into a second mobile application, and providing a control to initiate the action for the entity.
Abstract:
Systems and methods are provided for a personalized entity repository. For example, a computing device comprises a personalized entity repository having fixed sets of entities from an entity repository stored at a server, a processor, and memory storing instructions that cause the computing device to identify fixed sets of entities that are relevant to a user based on context associated with the computing device, rank the fixed sets by relevancy, and update the personalized entity repository using selected sets determined based on the rank and on set usage parameters applicable to the user. In another example, a method includes generating fixed sets of entities from an entity repository, including location-based sets and topic-based sets, and providing a subset of the fixed sets to a client, the client requesting the subset based on the client's location and on items identified in content generated for display on the client.
Abstract:
Implementations of the present disclosure include actions of receiving image data, the image data being provided from a camera and corresponding to a scene viewed by the camera, receiving one or more annotations, the one or more annotations being provided based on one or more entities determined from the scene, each annotation being associated with at least one entity, determining one or more actions based on the one or more annotations, and providing instructions to display an action interface including one or more action elements, each action element being selectable to induce execution of a respective action, the action interface being displayed in a viewfinder.
Abstract:
Systems and methods are provided for providing insight for entities in mobile onscreen content. For example, a method includes receiving, from a mobile device, an indication of selection of a first entity represented by a visual cue in first annotation data for a screen capture image of a screen of the mobile device and determining entities related to the first entity in a graph-based data store. The method may also include identifying a second entity in the screen capture image that is one of the entities related to the first entity, generating second annotation data, the second annotation data including a visual element linking the first entity and the second entity, and providing the second annotation data for display with the screen on the mobile device.
Abstract:
Systems and methods prevent or restrict the mining of content on a mobile device. For example, a method may include determining that content to be displayed on a screen includes content that matches a mining-restriction trigger, inserting a mining-restriction mark in the content that protects at least a portion of the content, and displaying the content with the mining-restriction mark on the screen. As another example, a method may include identifying, by a first application running on a mobile device, a mining-restriction mark in frame buffer data, the mining-restriction mark having been inserted by a second application, and determining whether the mining-restriction mark prevents mining of content. The method may also include preventing mining when the mining-restriction mark prevents mining and, when the mining-restriction mark does not prevent mining, determining a restriction for the data based on the mining-restriction mark and providing the restriction with the data for further processing.
Abstract:
Methods, systems, and apparatus include computer programs encoded on a computer-readable storage medium, including a method for providing content. Snapshots associated with use of a computing device by a user are received. Each snapshot is based on content presented to the user. The snapshots are evaluated. For each respective snapshot, a respective set of entities indicated by the respective snapshot is identified. Indications of the respective set of entities and a respective timestamp indicating a respective time that the respective snapshot was captured are associated and stored. Based on a first snapshot of the snapshots, a first time to present one or more information cards to the user is determined. At the first time, entities having a time stamp that corresponds to the first time are located. An information card is generated based on the located entities. The generated information card is provided for presentation to the user.
Abstract:
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing queries made up of images. In one aspect, a method includes indexing images by image descriptors. The method further includes associating descriptive n-grams with the images. In another aspect, a method includes receiving a query, identifying text describing the query, and performing a search according to the text identified for the query.