Abstract:
Implementations provide an improved drag-and-drop operation on a mobile device. For example, a method includes identifying a drag area in a user interface of a first mobile application in response to a drag command, identifying an entity from a data store based on recognition performed on content in the drag area, receiving a drop location associated with a second mobile application, determining an action to perform in the second mobile application based on the drop location, and performing the action in the second mobile action using the entity. Another method may include receiving a selection of a smart copy control for a text input control in a first mobile application, receiving a selected area of a display generated by a second mobile application, identifying an entity in the selected area, automatically navigating back to the text input control, and pasting a description of the entity in the text input control.
Abstract:
Systems and methods prevent or restrict the mining of content on a mobile device. For example, a method may include identifying a mining-restriction mark in low order bits or high order bits in a frame buffer of a mobile device and determining whether the mining-restriction mark prevents mining of content. Mining includes non-transient storage of a copy or derivations of data in the frame buffer. The method may also include preventing the mining of data in the frame buffer when the mining-restriction mark prevents mining.
Abstract:
A system and method of identifying objects is provided. In one aspect, the system and method includes a hand-held device with a display, camera and processor. As the camera captures images and displays them on the display, the processor compares the information retrieved in connection with one image with information retrieved in connection with subsequent images. The processor uses the result of such comparison to determine the object that is likely to be of greatest interest to the user. The display simultaneously displays the images the images as they are captured, the location of the object in an image, and information retrieved for the object.
Abstract:
A system and method of identifying objects is provided. In one aspect, the system and method includes a hand-held device with a display, camera and processor. As the camera captures images and displays them on the display, the processor compares the information retrieved in connection with one image with information retrieved in connection with subsequent images. The processor uses the result of such comparison to determine the object that is likely to be of greatest interest to the user. The display simultaneously displays the images the images as they are captured, the location of the object in an image, and information retrieved for the object.
Abstract:
Systems and methods are provided for sharing a screen from a mobile device. For example, a method includes receiving an image from a mobile device, performing recognition on the image to identify space-delimited strings, and generating a content graph for the image, the content graph having content nodes that represent at least some of the strings and the content graph having edges that represent a relative position of strings associated with the content nodes connected by the edges. The method may also include repeating the receiving, performing recognition, and generating for a plurality of images, the plurality of images belonging to a session, and generating a combined graph from the plurality of content graphs based on similarity of content nodes between content graphs, the combined graph representing text from the plurality of images in reading order.
Abstract:
A method, system, and computer readable storage medium is provided for identifying textual terms in response to a visual query is provided. A server system receives a visual query from a client system. The visual query is responded to as follows. A set of image feature values for the visual query is generated. The set of image feature values is mapped to a plurality of textual terms, including a weight for each of the textual terms in the plurality of textual terms. The textual terms are ranked in accordance with the weights of the textual terms. Then, in accordance with the ranking the textual terms, one or more of the ranked textual terms are sent to the client system.
Abstract:
Implementations provide an improved drag-and-drop operation on a mobile device. For example, a method includes identifying a drag area in a user interface of a first mobile application in response to a drag command, identifying an entity from a data store based on recognition performed on content in the drag area, receiving a drop location associated with a second mobile application, determining an action to perform in the second mobile application based on the drop location, and performing the action in the second mobile action using the entity. Another method may include receiving a selection of a smart copy control for a text input control in a first mobile application, receiving a selected area of a display generated by a second mobile application, identifying an entity in the selected area, automatically navigating back to the text input control, and pasting a description of the entity in the text input control.
Abstract:
A facial recognition search system identifies one or more likely names (or other personal identifiers) corresponding to the facial image(s) in a query as follows. After receiving the visual query with one or more facial images, the system identifies images that potentially match the respective facial image in accordance with visual similarity criteria. Then one or more persons associated with the potential images are identified. For each identified person, person-specific data comprising metrics of social connectivity to the requester are retrieved from a plurality of applications such as communications applications, social networking applications, calendar applications, and collaborative applications. An ordered list of persons is then generated by ranking the identified persons in accordance with at least metrics of visual similarity between the respective facial image and the potential image matches and with the social connection metrics. Finally, at least one person identifier from the list is sent to the requester.
Abstract:
An image input is obtained from a computing device when an image sensor of the computing device is directed to a scene. At least an object of interest in the scene is determined, and a label is determined for the object of interest. A search input is received from the computing device, where the search input is obtained from a mechanism other than the image sensor. An ambiguity is determined from the search input. A search query is determined that augments or replaces the ambiguity based at least in part on the label. A search result is based on the search query.
Abstract:
Systems and methods are provided for suggesting actions for selected text based on content displayed on a mobile device. An example method can include converting a selection made via a display device into a query, providing the query to an action suggestion model that is trained to predict an action given a query, each action being associated with a mobile application, receiving one or more predicted actions, and initiating display of the one or more predicted actions on the display device. Another example method can include identifying, from search records, queries where a website is highly ranked, the website being one of a plurality of websites in a mapping of websites to mobile applications. The method can also include generating positive training examples for an action suggestion model from the identified queries, and training the action suggestion model using the positive training examples.