Interpretability of deep reinforcement learning models in assistant systems

    公开(公告)号:US11715042B1

    公开(公告)日:2023-08-01

    申请号:US16389769

    申请日:2019-04-19

    摘要: In one embodiment, a method includes training a target machine-learning model iteratively by accessing training data of content objects, training an intermediate machine-learning model that outputs contextual evaluation measurements based on the training data, generating state-indications associated with the training data, wherein the state-indications comprise user-intents, system actions, and user actions, training the target machine-learning model based on the contextual evaluation measurements, the state-indications, and an action set comprising possible system actions, extracting rules based on the target machine-learning model by a sequential pattern-mining model, generating synthetic training data based on the rules, updating the training data by adding the synthetic training data to the training data, determining if a completion condition is reached for the training, and if the completion condition is reached returning the target machine-learning model, else repeating the iterative training of the target machine-learning model.

    Active Federated Learning for Assistant Systems

    公开(公告)号:US20240112008A1

    公开(公告)日:2024-04-04

    申请号:US16815960

    申请日:2020-03-11

    摘要: In one embodiment, a method includes receiving, by a first client system, from one or more remote servers, a current version of a neural network model including multiple model parameters, training the neural network model on multiple examples retrieved from a local data store to generate multiple updated model parameters, wherein each of the examples includes one or more features and one or more labels, calculating a user valuation associated with the first client system, wherein the user valuation represents a measure of utility of training the neural network model on the multiple examples, and sending, to one or more of the remote servers, the trained neural network model and the user valuation, wherein the user valuation is associated with a likelihood of the first client system being selected for a subsequent training of the neural network model.

    Conversational reasoning with knowledge graph paths for assistant systems

    公开(公告)号:US11442992B1

    公开(公告)日:2022-09-13

    申请号:US16557055

    申请日:2019-08-30

    摘要: In one embodiment, a method includes receiving a query from a user from a client system associated with the user, accessing a knowledge graph comprising a plurality of nodes and edges connecting the nodes, wherein each node corresponds to an entity and each edge corresponds to a relationship between the entities corresponding to the connected nodes, determining one or more initial entities associated with the query based on the query, selecting one or more candidate nodes by a conversational reasoning model from the knowledge graph corresponding to one or more candidate entities, respectively, wherein each candidate node is selected based on the nodes corresponding to the initial entities, dialog states associated with the query, and a context associated with the query, generating a response based on the initial entities and the candidate entities, and sending instructions for presenting the response to the client system in response to the query.

    Interpretability of deep reinforcement learning models in assistant systems

    公开(公告)号:US11657333B1

    公开(公告)日:2023-05-23

    申请号:US16389769

    申请日:2019-04-19

    摘要: In one embodiment, a method includes training a target machine-learning model iteratively by accessing training data of content objects, training an intermediate machine-learning model that outputs contextual evaluation measurements based on the training data, generating state-indications associated with the training data, wherein the state-indications comprise user-intents, system actions, and user actions, training the target machine-learning model based on the contextual evaluation measurements, the state-indications, and an action set comprising possible system actions, extracting rules based on the target machine-learning model by a sequential pattern-mining model, generating synthetic training data based on the rules, updating the training data by adding the synthetic training data to the training data, determining if a completion condition is reached for the training, and if the completion condition is reached returning the target machine-learning model, else repeating the iterative training of the target machine-learning model.