公开(公告)号：US20230401445A1

公开(公告)日：2023-12-14

申请号：US18457708

申请日：2023-08-29

Applicant: Microsoft Technology Licensing, LLC

Inventor： Dilek Z. Hakkani-Tur ,  Asli Celikyilmaz ,  Yun-Nung Chen ,  Li Deng ,  Jianfeng Gao ,  Gokhan Tur ,  Ye Yi Wang

IPC: G06N3/08 ,  G10L15/16 ,  G10L15/22 ,  G10L15/18 ,  G06N3/044

CPC classification number: G06N3/08 ,  G10L15/16 ,  G10L15/22 ,  G10L15/1822 ,  G06N3/044

Abstract: A processing unit can train a model as a joint multi-domain recurrent neural network (JRNN), such as a bi-directional recurrent neural network (bRNN) and/or a recurrent neural network with long-short term memory (RNN-LSTM) for spoken language understanding (SLU). The processing unit can use the trained model to, e.g., jointly model slot filling, intent determination, and domain classification. The joint multi-domain model described herein can estimate a complete semantic frame per query, and the joint multi-domain model enables multi-task deep learning leveraging the data from multiple domains. The joint multi-domain recurrent neural (JRNN) can leverage semantic intents (such as, finding or identifying, e.g., a domain specific goal) and slots (such as, dates, times, locations, subjects, etc.) across multiple domains.

公开(公告)号：US11715000B2

公开(公告)日：2023-08-01

申请号：US15639304

申请日：2017-06-30

Applicant: Microsoft Technology Licensing, LLC

Inventor： Fethiye Asli Celikyilmaz ,  Li Deng ,  Lihong Li ,  Chong Wang

IPC: G06N3/08 ,  G06F16/332 ,  G06N3/006

CPC classification number: G06N3/08 ,  G06F16/3329 ,  G06N3/006

Abstract: Systems and methods are disclosed for inquiry-based deep learning. In one implementation, a first content segment is selected from a body of content. The content segment includes a first content element. The first content segment is compared to a second content segment to identify a content element present in the first content segment that is not present in the second content segment. Based on an identification of the content element present in the first content segment that is not present in the second content segment, the content element is stored in a session memory. A first question is generated based on the first content segment. The session memory is processed to compute an answer to the first question. An action is initiated based on the answer. Using deep learning, content segments can be encoded into memory. Incremental questioning can serve to focus various deep learning operations on certain content segments.

公开(公告)号：US20190005385A1

公开(公告)日：2019-01-03

申请号：US15639304

申请日：2017-06-30

Applicant: Microsoft Technology Licensing, LLC

Inventor： Fethiye Asli Celikyilmaz ,  Li Deng ,  Lihong Li ,  Chong Wang

IPC: G06N3/08 ,  G06N3/00

Abstract: Systems and methods are disclosed for inquiry-based deep learning. In one implementation, a first content segment is selected from a body of content. The content segment includes a first content element. The first content segment is compared to a second content segment to identify a content element present in the first content segment that is not present in the second content segment. Based on an identification of the content element present in the first content segment that is not present in the second content segment, the content element is stored in a session memory. A first question is generated based on the first content segment. The session memory is processed to compute an answer to the first question. An action is initiated based on the answer. Using deep learning, content segments can be encoded into memory. Incremental questioning can serve to focus various deep learning operations on certain content segments.

公开(公告)号：US10089576B2

公开(公告)日：2018-10-02

申请号：US14811808

申请日：2015-07-28

Applicant: Microsoft Technology Licensing, LLC

Inventor： Jianfeng Gao ,  Li Deng ,  Xiaodong He ,  Ye-Yi Wang ,  Kevin Duh ,  Xiaodong Liu

IPC: G06F17/30 ,  G06N3/08

Abstract: A system may comprise one or more processors and memory storing instructions that, when executed by one or more processors, configure one or more processors to perform a number of operations or tasks, such as receiving a query or a document, and mapping the query or the document into a lower dimensional representation by performing at least one operational layer that shares at least two disparate tasks.

公开(公告)号：US20180157747A1

公开(公告)日：2018-06-07

申请号：US15367630

申请日：2016-12-02

Applicant: Microsoft Technology Licensing, LLC

Inventor： Saurabh Kumar Tiwary ,  Mir Rosenberg ,  Jianfeng Gao ,  Xia Song ,  Rangan Majumder ,  Li Deng

IPC: G06F17/30 ,  G06N3/08 ,  G06N5/04

CPC classification number: G06F16/951 ,  G06F16/3329 ,  G06F16/3338 ,  G06N3/08 ,  G06N5/04

Abstract: Systems and methods for automated generation of new content responses to answer user queries are provided. The systems and methods for automated generation of new content responses answer user queries utilizing deep learning and a reasoning algorithm. The generated response is composed of new content and is not merely cut or copied information from one or more search results. Accordingly, the systems and methods for automated generation of new content responses provide tailored query specific answers that can be long and detailed including several sentences of information or that can be short and concise, such as “yes” or “no.” The ability of the systems and methods described herein to create or generate new content in response to a user query improves the usability, improves the performance, and/or improves user interactions of/with a search query system.

公开(公告)号：US20180067923A1

公开(公告)日：2018-03-08

申请号：US15258639

申请日：2016-09-07

Applicant: Microsoft Technology Licensing, LLC

Inventor： Yun-Nung Chen ,  Dilek Z. Hakkani-Tur ,  Gokhan Tur ,  Asli Celikyilmaz ,  Jianfeng Gao ,  Li Deng

IPC: G06F17/27 ,  G06F17/21 ,  G10L25/30

CPC classification number: G06F17/2785 ,  G06F17/2705 ,  G10L15/16 ,  G10L15/1822

Abstract: Systems and methods for determining knowledge-guided information for a recurrent neural networks (RNN) to guide the RNN in semantic tagging of an input phrase are presented. A knowledge encoding module of a Knowledge-Guided Structural Attention Process (K-SAP) receives an input phrase and, in conjunction with additional sub-components or cooperative components generates a knowledge-guided vector that is provided with the input phrase to the RNN for linguistic semantic tagging. Generating the knowledge-guided vector comprises at least parsing the input phrase and generating a corresponding hierarchical linguistic structure comprising one or more discrete sub-structures. The sub-structures may be encoded into vectors along with attention weighting identifying those sub-structures that have greater importance in determining the semantic meaning of the input phrase.

公开(公告)号：US20170286494A1

公开(公告)日：2017-10-05

申请号：US15084366

申请日：2016-03-29

Applicant: Microsoft Technology Licensing, LLC

Inventor： Xiaodong He ,  Li Deng ,  Jianfeng Gao ,  Wen-tau Yih ,  Moontae Lee ,  Paul Smolensky

IPC: G06F17/30

Abstract: A processing unit can determine multiple representations associated with a statement, e.g., subject or predicate representations. In some examples, the representations can lack representation of semantics of the statement. The computing device can determine a computational model of the statement based at least in part on the representations. The computing device can receive a query, e.g., via a communications interface. The computing device can determine at least one query representation, e.g., a subject, predicate, or entity representation. The computing device can then operate the model using the query representation to provide a model output. The model output can represent a relationship between the query representations and information in the model. The computing device can, e.g., transmit an indication of the model output via the communications interface. The computing device can determine mathematical relationships between subject representations and attribute representations for multiple statements, and determine the model using the relationships.

公开(公告)号：US20170193360A1

公开(公告)日：2017-07-06

申请号：US14985017

申请日：2015-12-30

Applicant: Microsoft Technology Licensing, LLC

Inventor： Jianfeng Gao ,  Li Deng ,  Xiaodong He ,  Prabhdeep Singh ,  Lihong Li ,  Jianshu Chen ,  Xiujun Li ,  Ji He

IPC: G06N3/08

CPC classification number: G06N3/08 ,  G06N3/0445 ,  G06N3/0454

Abstract: A processing unit can operate a first recurrent computational model (RCM) to provide first state information and a predicted result value. The processing unit can operating a first network computational model (NCM) to provide respective expectation values of a plurality of actions based at least in part on the first state information. The processing unit can provide an indication of at least one of the plurality of actions, and receive a reference result value, e.g., via a communications interface. The processing unit can train the first RCM based at least in part on the predicted result value and the reference result value to provide a second RCM, and can train the first NCM based at least in part on the first state information and the at least one of the plurality of actions to provide a second NCM.

公开(公告)号：US20170147942A1

公开(公告)日：2017-05-25

申请号：US14949156

申请日：2015-11-23

Applicant: Microsoft Technology Licensing, LLC

Inventor： Jianfeng Gao ,  Li Deng ,  Xiaodong He ,  Lin Xiao ,  Xinying Song ,  Yelong Shen ,  Ji He ,  Jianshu Chen

IPC: G06N99/00

CPC classification number: G06N7/005

Abstract: A processing unit can successively operate layers of a multilayer computational graph (MCG) according to a forward computational order to determine a topic value associated with a document based at least in part on content values associated with the document. The processing unit can successively determine, according to a reverse computational order, layer-specific deviation values associated with the layers based at least in part on the topic value, the content values, and a characteristic value associated with the document. The processing unit can determine a model adjustment value based at least in part on the layer-specific deviation values. The processing unit can modify at least one parameter associated with the MCG based at least in part on the model adjustment value. The MCG can be operated to provide a result characteristic value associated with test content values of a test document.

公开(公告)号：US11449744B2

公开(公告)日：2022-09-20

申请号：US15229039

申请日：2016-08-04

Applicant: Microsoft Technology Licensing, LLC

Inventor： Yun-Nung Chen ,  Dilek Z. Hakkani-Tur ,  Gokhan Tur ,  Li Deng ,  Jianfeng Gao

IPC: G06N3/08 ,  G06N3/04 ,  G10L15/16 ,  G10L15/18 ,  G10L15/22

Abstract: A processing unit can extract salient semantics to model knowledge carryover, from one turn to the next, in multi-turn conversations. Architecture described herein can use the end-to-end memory networks to encode inputs, e.g., utterances, with intents and slots, which can be stored as embeddings in memory, and in decoding the architecture can exploit latent contextual information from memory, e.g., demographic context, visual context, semantic context, etc. e.g., via an attention model, to leverage previously stored semantics for semantic parsing, e.g., for joint intent prediction and slot tagging. In examples, architecture is configured to build an end-to-end memory network model for contextual, e.g., multi-turn, language understanding, to apply the end-to-end memory network model to multiple turns of conversational input; and to fill slots for output of contextual, e.g., multi-turn, language understanding of the conversational input. The neural network can be learned using backpropagation from output to input using gradient descent optimization.