-
公开(公告)号:US09916328B1
公开(公告)日:2018-03-13
申请号:US14451393
申请日:2014-08-04
Applicant: GOOGLE INC.
Inventor: Matthew Sharifi , David Petrou
CPC classification number: G06F17/30259 , G06F3/04842 , G06F3/04845 , G06F9/451 , G06F17/2235 , G06F17/241 , G06F17/278 , G06F17/30112 , G06F17/30247 , G06F17/30268 , G06F17/30321 , G06F17/3053 , G06F17/30899 , G06K9/00469 , G06K9/6215 , G06Q10/1097 , H04L67/06 , H04W4/18
Abstract: Systems and methods are provided for providing user assistance on a mobile device. For example, a method may include receiving a first image of a screen captured on a mobile device, determining key content in the first image by performing recognition on the first image, and using the key content as a query to an index that associates key items with screen capture images previously received from the mobile device. The method may further include determining that a search result of the query includes at least one key item with a rank that exceeds a relevancy threshold, determining a portion of a second image from the screen capture images that is associated with the at least one key item, generating annotation data that includes the portion of the second image, and providing the annotation data for display with the screen on the mobile device.
-
公开(公告)号:US09886461B1
公开(公告)日:2018-02-06
申请号:US14451390
申请日:2014-08-04
Applicant: GOOGLE INC.
Inventor: Matthew Sharifi , David Petrou
IPC: G06F17/30
CPC classification number: G06F17/30259 , G06F3/04842 , G06F3/04845 , G06F9/451 , G06F17/2235 , G06F17/241 , G06F17/278 , G06F17/30112 , G06F17/30247 , G06F17/30268 , G06F17/30321 , G06F17/3053 , G06F17/30899 , G06K9/00469 , G06K9/6215 , G06Q10/1097 , H04L67/06 , H04W4/18
Abstract: Systems and methods for indexing onscreen content on a mobile device are provided. For example, a method can include determining, using an index for a user that associates key items with images of screens captured from a mobile device of the user, that a first key item is responsive to a query initiated at the mobile device, and identifying responsive images, the responsive images being associated with the first key item. The method may also include highlighting, in the responsive images, areas corresponding to the first key item, and providing a search result to the mobile device that includes portions of the responsive images. The portions of the responsive images may be areas corresponding to the first key item in the images.
-
公开(公告)号:US09792304B1
公开(公告)日:2017-10-17
申请号:US14946027
申请日:2015-11-19
Applicant: Google Inc.
Inventor: Ulrich Buddemeier , Gabriel Taubman , Hartwig Adam , Charles J. Rosenberg , Hartmut Neven , David Petrou , Fernando Brucher
IPC: G06F17/30
CPC classification number: G06F17/30277 , G06F17/30256 , G06F17/30268 , G06F17/3053 , G06F17/30554 , G06K9/6215 , G06K9/6282 , G06K9/723 , G06K2209/01
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing queries made up of images. In one aspect, a method includes indexing images by image descriptors. The method further includes associating descriptive n-grams with the images. In another aspect, a method includes receiving a query, identifying text describing the query, and performing a search according to the text identified for the query.
-
公开(公告)号:US09788179B1
公开(公告)日:2017-10-10
申请号:US14712679
申请日:2015-05-14
Applicant: GOOGLE INC.
Inventor: Matthew Sharifi , David Petrou
CPC classification number: G06F17/30259 , G06F3/04842 , G06F3/04845 , G06F9/451 , G06F17/2235 , G06F17/241 , G06F17/278 , G06F17/30112 , G06F17/30247 , G06F17/30268 , G06F17/30321 , G06F17/3053 , G06F17/30899 , G06K9/00469 , G06K9/6215 , G06Q10/1097 , H04L67/06 , H04W4/18
Abstract: Systems and methods are provided for detecting and ranking entities identified in screen content displayed on a mobile device. For example, a method includes receiving an image captured from a mobile device display for a mobile application and determining a window that includes a chronological set of images, the images each representing a respective screen captured from a display of a mobile device and having an associated timestamp. The method also includes identifying entities appearing in images in a first portion of the window using text for images in a remaining portion of the window as context to disambiguate ambiguous entity references.
-
公开(公告)号:US09684693B1
公开(公告)日:2017-06-20
申请号:US15091193
申请日:2016-04-05
Applicant: Google Inc.
Inventor: David Petrou , Matthew Sharifi
CPC classification number: G06F17/30864 , G06F17/278 , G06F17/30672
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, relating to on-device query annotating. In some implementations, a search query is received, and a mobile device identifies a reference to a particular entity and a reference to a category based on the query. A model that is stored on the mobile device and stores one or more facts that are associated with one or more entities is accessed. A subset of facts from among the facts that are stored in the model for the particular entity is selected. The search query is annotated based at least on one or more facts of the subset of facts that are stored in the model for the particular entity. The annotated search query is transmitted, from the mobile device to a search engine, for processing. A result of processing the annotated search query is received by the mobile device.
-
公开(公告)号:US20170098159A1
公开(公告)日:2017-04-06
申请号:US14872582
申请日:2015-10-01
Applicant: GOOGLE INC.
Inventor: Matthew Sharifi , Daniel Ramage , David Petrou
IPC: G06N5/02 , G06F17/30 , G06F3/0484
CPC classification number: G06N5/02 , G06F3/0481 , G06F3/0482 , G06F3/0484 , G06F3/04895 , G06F9/453 , G06F16/245 , G06N20/00
Abstract: Systems and methods are provided for suggesting actions for selected text based on content displayed on a mobile device. An example method can include converting a selection made via a display device into a query, providing the query to an action suggestion model that is trained to predict an action given a query, each action being associated with a mobile application, receiving one or more predicted actions, and initiating display of the one or more predicted actions on the display device. Another example method can include identifying, from search records, queries where a website is highly ranked, the website being one of a plurality of websites in a mapping of websites to mobile applications. The method can also include generating positive training examples for an action suggestion model from the identified queries, and training the action suggestion model using the positive training examples.
-
公开(公告)号:US20150227797A1
公开(公告)日:2015-08-13
申请号:US14618962
申请日:2015-02-10
Applicant: GOOGLE INC.
Inventor: Teresa Ko , Hartwig Adam , Mikkel Crone Koser , Alexei Masterov , Andrews-Junior Kimbembe , Matthew J. Bridges , Paul Chang , David Petrou
CPC classification number: H04N5/23293 , G06F3/04817 , G06F3/04842 , G06F3/1431 , G06F3/1438 , G06K9/00671 , G06K9/00677 , G06K9/78 , G09G5/363 , G09G2360/08 , H04N5/23216
Abstract: Implementations of the present disclosure include actions of receiving image data, the image data being provided from a camera and corresponding to a scene viewed by the camera, receiving one or more annotations, the one or more annotations being provided based on one or more entities determined from the scene, each annotation being associated with at least one entity, determining one or more actions based on the one or more annotations, and providing instructions to display an action interface including one or more action elements, each action element being selectable to induce execution of a respective action, the action interface being displayed in a viewfinder.
Abstract translation: 本公开的实现包括接收图像数据的动作,图像数据从相机提供并对应于由相机观看的场景,接收一个或多个注释,基于确定的一个或多个实体提供的一个或多个注释 从场景中,每个注释与至少一个实体相关联,基于一个或多个注释来确定一个或多个动作,以及提供指示以显示包括一个或多个动作元素的动作界面,每个动作元素可选择以诱导执行 操作界面被显示在取景器中。
-
公开(公告)号:US20140172881A1
公开(公告)日:2014-06-19
申请号:US14185392
申请日:2014-02-20
Applicant: Google Inc.
Inventor: David Petrou , Andrew Rabinovich , Hartwig Adam
IPC: G06F17/30
CPC classification number: G06F17/30268 , G06F17/30256 , G06F17/30277 , G06F17/3053 , G06F17/30867 , G06K9/00288 , G06K9/00677
Abstract: A facial recognition search system identifies one or more likely names (or other personal identifiers) corresponding to the facial image(s) in a query as follows. After receiving the visual query with one or more facial images, the system identifies images that potentially match the respective facial image in accordance with visual similarity criteria. Then one or more persons associated with the potential images are identified. For each identified person, person-specific data comprising metrics of social connectivity to the requester are retrieved from a plurality of applications such as communications applications, social networking applications, calendar applications, and collaborative applications. An ordered list of persons is then generated by ranking the identified persons in accordance with at least metrics of visual similarity between the respective facial image and the potential image matches and with the social connection metrics. Finally, at least one person identifier from the list is sent to the requester.
Abstract translation: 面部识别搜索系统如下所述识别与查询中的面部图像相对应的一个或多个可能的名称(或其他个人标识符)。 在接收到具有一个或多个面部图像的视觉查询之后,系统根据视觉相似性标准识别潜在地匹配相应面部图像的图像。 然后识别与潜在图像相关联的一个或多个人。 对于每个已识别的人,从诸如通信应用,社交网络应用,日历应用和协作应用的多个应用中检索包括对请求者的社交连接性度量的个人特定数据。 然后通过根据相应的面部图像和潜在图像匹配之间的视觉相似性的度量以及与社会连接度量进行评估来对所识别的人进行排序来生成人的有序列表。 最后,从列表中至少有一个人标识符被发送到请求者。
-
公开(公告)号:US20140164406A1
公开(公告)日:2014-06-12
申请号:US14183001
申请日:2014-02-18
Applicant: GOOGLE INC.
Inventor: David Petrou
IPC: G06F17/30
Abstract: A visual query such as a photograph, a screen shot, a scanned image, a video frame, or an image created by a content authoring application is submitted to a visual query search system. The search system processes the visual query by sending it to a plurality of parallel search systems, each implementing a distinct visual query search process. These parallel search systems may include but are not limited to optical character recognition (OCR), facial recognition, product recognition, bar code recognition, object-or-object-category recognition, named entity recognition, and color recognition. Then at least one search result is sent to the client system. In some embodiments, when the visual query is an image containing a text element and a non-text element, at least one search result includes an optical character recognition result for the text element and at least one image-match result for the non-text element.
Abstract translation: 诸如照片,屏幕截图,扫描图像,视频帧或由内容创作应用创建的图像的视觉查询被提交到视觉查询搜索系统。 搜索系统通过将视觉查询发送到多个并行搜索系统来处理视觉查询,每个并行搜索系统实现不同的可视查询搜索过程。 这些并行搜索系统可以包括但不限于光学字符识别(OCR),面部识别,产品识别,条形码识别,对象或对象类别识别,命名实体识别和颜色识别。 然后至少一个搜索结果发送到客户端系统。 在一些实施例中,当视觉查询是包含文本元素和非文本元素的图像时,至少一个搜索结果包括文本元素的光学字符识别结果和非文本元素的至少一个图像匹配结果 元件。
-
公开(公告)号:US10241668B2
公开(公告)日:2019-03-26
申请号:US15432493
申请日:2017-02-14
Applicant: GOOGLE INC.
Inventor: Matthew Sharifi , David Petrou
IPC: G06F3/0486 , G06F3/16 , G06F9/54 , G06F3/0484 , G06F3/0488
Abstract: Implementations provide an improved drag-and-drop operation on a mobile device. For example, a method includes identifying a drag area in a user interface of a first mobile application in response to a drag command and receiving a drop location in a second mobile application that differs from the first mobile application. The method may also include determining that a drop location is a text input control and the drag area is not text-based, performing a search for a text description of the drag area, and pasting the text description into the text input control. The method may also include determining that a drop location is an image input control and that the drag area is text based, performing a search using the drag area for a responsive image, and pasting the responsive image into the image input control.
-
-
-
-
-
-
-
-
-