公开(公告)号：US12111837B1

公开(公告)日：2024-10-08

申请号：US18306869

申请日：2023-04-25

申请人： Microsoft Technology Licensing, LLC

发明人： Jian Jiao ,  Yeyun Gong ,  Xingwei He ,  Nan Duan ,  Eren Manavoglu

IPC分类号： G06F16/2457 ,  G06F16/242 ,  G06F16/248 ,  G06N20/00

CPC分类号： G06F16/24578 ,  G06F16/2438 ,  G06F16/248 ,  G06N20/00

摘要： Technologies described herein relate to dense retrieval and ranking of search results. A query indicating a computing context or user input is received. An embedding of the query is computed by way of a first encoder, and candidate results selected from a pool of potential results based upon the embedding of the query and embeddings of the potential results. A similarity score for a first of the candidate results is computed by way of a second encoder trained based upon an order metric that defines a ranking over a training set of potential results. The first encoder is trained based upon output of the second encoder prior to computing the embedding of the query. The candidate results are ranked based upon the similarity score of the first candidate result, and results responsive to the query are identified based upon the ranking. The identified results are output to a computing device.

公开(公告)号：US11461415B2

公开(公告)日：2022-10-04

申请号：US16784200

申请日：2020-02-06

申请人： Microsoft Technology Licensing, LLC

发明人： Wenhao Lu ,  Jian Jiao ,  Ruofei Zhang

IPC分类号： G06F16/953 ,  G06F40/284 ,  G06F40/289 ,  G06F40/30 ,  G06N3/08 ,  G06Q30/02

摘要： A technique is described herein for processing a given query item in a latency-efficient and resource-efficient manner. The technique uses a first transformer-based encoder to transform the given query item into an encoded query item. In one case, the given query item is an expression that includes one or more query-expression linguistic tokens. The technique includes a second transformer-based encoder for transforming a given target item into an encoded target item. The given target item may likewise correspond to an expression that includes one or more target-expression linguistic tokens. A similarity-assessing mechanism then assesses the semantic similarity between the given query item and the given target item based on the encoded query item and the encoded target item. Each transformer-based encoder uses one or more self-attention mechanisms. The second transformer-based encoder can optionally perform its work in an offline manner, prior to receipt of the given query item.

公开(公告)号：US20170236056A1

公开(公告)日：2017-08-17

申请号：US15226196

申请日：2016-08-02

申请人： Microsoft Technology Licensing, LLC

发明人： Ying Shan ,  Thomas Ryan Hoens ,  Jian Jiao ,  Haijing Wang ,  Dong Yu ,  JC Mao

IPC分类号： G06N3/08 ,  G06F17/30

摘要： Systems and methods for providing a predictive framework are provided. The predictive framework comprises plural neural layers of adaptable, executable neurons. Neurons accept one or more input signals and produce an output signal that may be used by an upper-level neural layer. Input signals are received by an encoding neural layer, where there is a 1:1 correspondence between an input signal and an encoding neuron. Input signals for a set of data are received at the encoding layer and processed successively by the plurality of neural layers. An objective function utilizes the output signals of the topmost neural layer to generate predictive results for the data set according to an objective. In one embodiment, the objective is to determine the likelihood of user interaction with regard to a specific item of content in a set of search results, or the likelihood of user interaction with regard to any item of content in a set of search results.

公开(公告)号：US10685281B2

公开(公告)日：2020-06-16

申请号：US15226196

申请日：2016-08-02

申请人： Microsoft Technology Licensing, LLC

发明人： Ying Shan ,  Thomas Ryan Hoens ,  Jian Jiao ,  Haijing Wang ,  Dong Yu ,  JC Mao

IPC分类号： G06N3/08 ,  G06F16/951 ,  G06Q30/02 ,  G06Q10/04 ,  G06N3/04

摘要： Systems and methods for providing a predictive framework are provided. The predictive framework comprises plural neural layers of adaptable, executable neurons. Neurons accept one or more input signals and produce an output signal that may be used by an upper-level neural layer. Input signals are received by an encoding neural layer, where there is a 1:1 correspondence between an input signal and an encoding neuron. Input signals for a set of data are received at the encoding layer and processed successively by the plurality of neural layers. An objective function utilizes the output signals of the topmost neural layer to generate predictive results for the data set according to an objective. In one embodiment, the objective is to determine the likelihood of user interaction with regard to a specific item of content in a set of search results, or the likelihood of user interaction with regard to any item of content in a set of search results.

公开(公告)号：US11023473B2

公开(公告)日：2021-06-01

申请号：US16017817

申请日：2018-06-25

申请人： Microsoft Technology Licensing, LLC

发明人： Ying Shan ,  Jian Jiao ,  Jie Zhu ,  Jianchang Mao

IPC分类号： G06F16/2457 ,  G06F7/16 ,  G06N3/04 ,  G06N3/063 ,  G06N3/08 ,  G06F16/2455

摘要： A computational search method for retrieving computer information related to a query includes transforming a plurality of candidate answers to candidate answer recurrent binary embedding (RBE) embeddings using a trained RBE model. A query is transformed to a query RBE embedding using the trained RBE model. The query RBE embedding is compared to each candidate answer RBE embedding of a plurality of candidate answer RBE embeddings using a similarity function. The candidate answers are sorted based on the comparisons made using the similarity function, and returning a plurality of the top candidate answers.

公开(公告)号：US11676001B2

公开(公告)日：2023-06-13

申请号：US17093426

申请日：2020-11-09

申请人： Microsoft Technology Licensing, LLC

发明人： Jian Jiao ,  Xiaodong Liu ,  Ruofei Zhang ,  Jianfeng Gao

IPC分类号： G06N3/045

CPC分类号： G06N3/045

摘要： Knowledge graphs can greatly improve the quality of content recommendation systems. There is a broad variety of knowledge graphs in the domain including clicked user-ad graphs, clicked query-ad graphs, keyword-display URL graphs etc. A hierarchical Transformer model learns entity embeddings in knowledge graphs. The model consists of two different Transformer blocks where the bottom block generates relation-dependent embeddings for the source entity and its neighbors, and the top block aggregates the outputs from the bottom block to produce the target entity embedding. To balance the information from contextual entities and the source entity itself, a masked entity model (MEM) task is combined with a link prediction task in model training.

公开(公告)号：US11966428B2

公开(公告)日：2024-04-23

申请号：US17364919

申请日：2021-07-01

申请人： Microsoft Technology Licensing, LLC

发明人： Jian Jiao ,  Yeyun Gong ,  Nan Duan ,  Ruofei Zhang

IPC分类号： G06F16/33 ,  G06F40/284 ,  G06N3/045 ,  G06N3/08 ,  G06N5/025 ,  G06N5/04

CPC分类号： G06F16/3334 ,  G06F40/284 ,  G06N3/045 ,  G06N3/08 ,  G06N5/025 ,  G06N5/04

摘要： A training system produces a resource-efficient machine-trained model via a training architecture that employs plural processing paths. Some of the processing paths incorporate the use of auxiliary information that imparts external knowledge about source items being processed. The training architecture also employs contrastive learning that operates at different respective levels within the training architecture. For instance, the training architecture uses encoder-level contrastive learning to compare output information generated by different encoders within the training architecture. The training architecture uses decoder-level contrastive learning to compare output information produced by different decoders within the training architecture. An inference-stage system performs an application task using the model produced by the training system.