公开(公告)号：US11036724B2

公开(公告)日：2021-06-15

申请号：US16560942

申请日：2019-09-04

Applicant: Microsoft Technology Licensing, LLC

Inventor： Li Huang ,  Houdong Hu ,  Meenaz Merchant ,  Arun Sacheti

IPC: G06F16/242 ,  G06F16/9038 ,  G06F3/0482 ,  G06F16/28 ,  G06F16/951 ,  G06F16/9535 ,  G06F3/0481

Abstract: A visual search engine is described herein. The visual search engine is configured to return information to a client computing device based upon a multimodal query received from the client computing device (wherein the multimodal query comprises an image and text). The visual search engine is further configured to interact with a user of the client computing device to disambiguate information retrieval intent of the user.

公开(公告)号：US10902052B2

公开(公告)日：2021-01-26

申请号：US15935521

申请日：2018-03-26

Applicant: MICROSOFT TECHNOLOGY LICENSING, LLC

Inventor： Mark Robert Bolin ,  Ning Ma ,  Aleksandr Livshits ,  Alexey Volkov ,  Pawel Michal Pietrusinksi ,  Houdong Hu

IPC: G06F16/248 ,  G06F16/58 ,  G06N3/08 ,  G06F7/02

Abstract: Systems and methods for identifying search results in response to a search query are presented. More particularly, images are selected as search results, at least in part, according to an attractiveness value associated with the images. Upon receiving a search query, a set of content is identified according to the query intent of the search query and includes at least one image. The identified set of content is ordered according an overall score determined according to relevance and, in the case of the at least one image, according to an attractiveness value. A search results generator selects items from the set of content according to their overall scores, including the at least one image, generates a search results page, and returns the search results page to the requesting party.

公开(公告)号：US20200019628A1

公开(公告)日：2020-01-16

申请号：US16036224

申请日：2018-07-16

Applicant: Microsoft Technology Licensing, LLC

Inventor： Xi Chen ,  Houdong Hu ,  Li Huang ,  Jiapei Huang ,  Arun Sacheti ,  Linjun Yang ,  Rui Xia ,  Kuang-Huei Lee ,  Meenaz Merchant ,  Sean Chang Culatana

IPC: G06F17/30 ,  G06K9/62 ,  G06N5/04 ,  G06N99/00 ,  G06F17/11

Abstract: Representative embodiments disclose mechanisms to perform visual intent classification or visual intent detection or both on an image. Visual intent classification utilizes a trained machine learning model that classifies subjects in the image according to a classification taxonomy. The visual intent classification can be used as a pre-triggering mechanism to initiate further action in order to substantially save processing time. Example further actions include user scenarios, query formulation, user experience enhancement, and so forth. Visual intent detection utilizes a trained machine learning model to identify subjects in an image, place a bounding box around the image, and classify the subject according to the taxonomy. The trained machine learning model utilizes multiple feature detectors, multi-layer predictions, multilabel classifiers, and bounding box regression.

公开(公告)号：US20190311070A1

公开(公告)日：2019-10-10

申请号：US15947564

申请日：2018-04-06

Applicant: Microsoft Technology Licensing, LLC

Inventor： Li Huang ,  Houdong Hu ,  Meenaz Merchant

IPC: G06F17/30 ,  G06K9/32 ,  G10L15/26 ,  G06N5/02

Abstract: A method for using a speech signal to augment a visual search includes processing the image data to determine an image search intent. Concurrently with processing the image data, the method processes the speech signal to determine at least one speech search intent. The method generates a search query by combining keywords and/or the image from the image search intent with keywords from the speech search intent. The method then performs a search based on the generated query and reports the results of the search. The method generates the image search intent by applying the image data to a knowledge base and generates the speech search intent by converting the speech to text and applying the text to a cognition service.

公开(公告)号：US20190236487A1

公开(公告)日：2019-08-01

申请号：US15883686

申请日：2018-01-30

Applicant: Microsoft Technology Licensing, LLC

Inventor： Jiapei Huang ,  Houdong Hu ,  Li Huang ,  Xi Chen ,  Linjun Yang

IPC: G06N99/00

CPC classification number: G06N20/00 ,  G06F3/04842

Abstract: A technique for hyperparameter tuning can be performed via a hyperparameter tuning tool. In the technique, computer-readable values for each of one or more machine learning hyperparameters can be received. Multiple computer-readable hyperparameter value sets can be defined using different combinations of the values. In response to a request to start, an overall hyperparameter tuning operation can be performed via the tool, with the overall operation including a tuning job for each of the hyperparameter sets. A computer-readable comparison of the results of the parameter tuning operations can be generated for the hyperparameter sets, with the comparison indicating effectiveness of the hyperparameter sets, as compared to each other, in the tuning jobs.

公开(公告)号：US12266034B2

公开(公告)日：2025-04-01

申请号：US17877935

申请日：2022-07-30

Applicant: Microsoft Technology Licensing, LLC

Inventor： Julia Gong ,  Houdong Hu ,  William Douglas Guyman

IPC: G06T11/00 ,  G06F16/53 ,  G06F16/583 ,  G06F40/20 ,  G06F40/30 ,  G06T7/00 ,  G06T7/90

Abstract: A data processing system implements a receiving a textual input comprising a query for a first image. The data processing system also implements analyzing the textual input to determine a predicted color palette associated with a subject matter of the query; and procedurally generating the first image using the predicted color palette. Another implementation of the data processing system implements providing the textual input to a first machine learning model to obtain the first image, the first machine learning model being trained using a dataset comprising abstract imagery and analyzing the textual input using the first machine learning model to obtain the first image in response to receiving the textual input.

公开(公告)号：US12216705B2

公开(公告)日：2025-02-04

申请号：US17404367

申请日：2021-08-17

Applicant: Microsoft Technology Licensing, LLC

Inventor： Li Huang ,  Meenaz Merchant ,  Houdong Hu ,  Arun Sacheti

IPC: G06F16/583 ,  G06F16/2457 ,  G06F16/248 ,  G06F16/51 ,  G06F16/532 ,  G06F16/56 ,  G06F18/22 ,  G06N3/04 ,  G06N3/08

Abstract: A visual search system includes a computing device, where the computing device includes an image processing engine for generating a feature vector representing a user-selected object in an image. The computing device also includes, an object detection engine for locating one or more objects in the image and for determining a category of a user-selected object from objects in the image, where the object detection engine uses the category to generate a plurality of attributes for the user-selected object. The computing device further includes a product data store for storing a plurality of tables storing one or more attributes associated with a category of the user-selected object. The computing device additionally includes an attribute generation engine for generating a plurality of attribute options and an attribute matching engine for comparing attributes and attribute options of the user-selected object with attributes and attribute options of visually similar products and images.

公开(公告)号：US11830056B2

公开(公告)日：2023-11-28

申请号：US17102009

申请日：2020-11-23

Applicant: Microsoft Technology Licensing, LLC

Inventor： Julia X. Gong ,  Jyotkumar Patel ,  Yale Song ,  Xuetao Yin ,  Xiujia Guo ,  Rajiv S. Binwade ,  Houdong Hu

IPC: G06Q30/0282 ,  G06F18/22 ,  G06Q30/0601 ,  G06N3/08 ,  G06Q30/0204 ,  G06N3/04 ,  G06Q50/00 ,  G06V10/32

CPC classification number: G06Q30/0631 ,  G06F18/22 ,  G06N3/04 ,  G06N3/08 ,  G06Q30/0205 ,  G06Q30/0282 ,  G06Q30/0639 ,  G06Q50/01 ,  G06V10/32

Abstract: The present disclosure provides method and apparatus for determining a food item from a photograph and a corresponding restaurant serving the food item. An image is received from a user, the image being associated with a consumable item. One or more ingredients of the consumable item in the image is identified along with a location of the user and using a neural network, determining one or more similar images from a database. A restaurant associated with each of the one or more similar images is determined along with a similarity score indicating a similarity between the restaurant and the identified content of the image. The one or more restaurants and/or associated similar food items are ranked based on the similarity score and a list of ranked restaurants is provided to the user.

公开(公告)号：US20210382935A1

公开(公告)日：2021-12-09

申请号：US17404367

申请日：2021-08-17

Applicant: Microsoft Technology Licensing, LLC

Inventor： Li Huang ,  Meenaz Merchant ,  Houdong Hu ,  Arun Sacheti

IPC: G06F16/532 ,  G06F16/51 ,  G06F16/2457 ,  G06F16/248 ,  G06F16/56

Abstract: A visual search system comprised of a computing device, the computing device including an image processing engine for generating a feature vector representing a user-selected object in an image input, an object detection engine for locating one or more objects in the image input and for determining a category of a user-selected object from objects in the image input, the object detection engine using the category to generate a plurality of attributes for the user-selected object, a product data store for storing a plurality of tables storing one or more attributes associated with a category of the user-selected object, an attribute generation engine for generating a plurality of attribute options for each of the attributes of the user-selected object, and an attribute matching engine for comparing attributes and attribute options of the user-selected object with attributes and attribute options of visually similar products and images.

公开(公告)号：US11074289B2

公开(公告)日：2021-07-27

申请号：US15885568

申请日：2018-01-31

Applicant: Microsoft Technology Licensing, LLC.

Inventor： Houdong Hu ,  Yan Wang ,  Linjun Yang ,  Li Huang ,  Xi Chen ,  Jiapei Huang ,  Ye Wu ,  Arun K. Sacheti ,  Meenaz Merchant

IPC: G06F16/53 ,  G06F16/532 ,  G06T7/00 ,  G06K9/62 ,  G06K9/46 ,  G06N3/08 ,  G06F16/51 ,  G06F16/56 ,  G06F16/583 ,  G06F16/2457

Abstract: Systems and methods can be implemented to conduct searches based on images used as queries in a variety of applications. In various embodiments, a set of visual words representing a query image are generated from features extracted from the query image and are compared with visual words of index images. A set of candidate images is generated from the index images resulting from matching one or more visual words in the comparison. A multi-level ranking is conducted to sort the candidate images of the set of candidate images, and results of the multi-level ranking are returned to a user device that provided the query image. Additional systems and methods are disclosed.