中科微点-全球专利检索

  1. Login
  2. Sign Up

Search Results

8 records (took 0.013 seconds)

    UTILIZING ELASTIC WEIGHT CONSOLIDATION (EWC) LOSS TERM(S) TO MITIGATE CATASTROPHIC FORGETTING IN TRAINING MACHINE LEARNING MODEL(S)
    1.
    发明申请
    UTILIZING ELASTIC WEIGHT CONSOLIDATION (EWC) LOSS TERM(S) TO MITIGATE CATASTROPHIC FORGETTING IN TRAINING MACHINE LEARNING MODEL(S) 有权

    公开(公告)号：US20250045627A1

    公开(公告)日：2025-02-06

    申请号：US18365487

    申请日：2023-08-04

    Applicant: GOOGLE LLC

    Inventor： Andrew Hard ,  Kurt Partridge ,  Sean Augenstein ,  Rajiv Mathews

    IPC: G06N20/00

    Abstract: Processor(s) of a client device can receive global weights of a global ML model from a remote system, obtain a client device data set, determine a Fisher information matrix for the client data set, and transmit the Fisher information matrix for the client data set to the remote system. Further, processor(s) of the remote system can determine a corresponding elastic weight consolidation (EWC) loss term for each of the global weights based on at least the Fisher information matrix, generate a server update for the global ML model based on (i) processing server data remotely at the remote system and using the global ML model and (ii) based on the corresponding EWC loss term for each of the global weights, and update the global weights of the global ML model based on the server update.

    USING CORRECTIONS, OF AUTOMATED ASSISTANT FUNCTIONS, FOR TRAINING OF ON-DEVICE MACHINE LEARNING MODELS
    2.
    发明公开
    USING CORRECTIONS, OF AUTOMATED ASSISTANT FUNCTIONS, FOR TRAINING OF ON-DEVICE MACHINE LEARNING MODELS 审中-公开

    公开(公告)号：US20230352019A1

    公开(公告)日：2023-11-02

    申请号：US18218818

    申请日：2023-07-06

    Applicant: GOOGLE LLC

    Inventor： Françoise Beaufays ,  Rajiv Mathews ,  Dragan Zivkovic ,  Kurt Partridge ,  Andrew Hard

    IPC: G10L15/22 ,  G10L15/065 ,  G10L15/10 ,  G10L15/30

    CPC classification number: G10L15/22 ,  G10L15/065 ,  G10L15/10 ,  G10L15/30

    Abstract: Processor(s) of a client device can: receive sensor data that captures environmental attributes of an environment of the client device; process the sensor data using a machine learning model to generate a predicted output that dictates whether one or more currently dormant automated assistant functions are activated; making a decision as to whether to trigger the one or more currently dormant automated assistant functions; subsequent to making the decision, determining that the decision was incorrect; and in response to determining that the determination was incorrect, generating a gradient based on comparing the predicted output to ground truth output. In some implementations, the generated gradient is used, by processor(s) of the client device, to update weights of the on-device speech recognition model. In some implementations, the generated gradient is additionally or alternatively transmitted to a remote system for use in remote updating of global weights of a global speech recognition model.

    Using corrections, of automated assistant functions, for training of on-device machine learning models
    3.
    发明授权
    Using corrections, of automated assistant functions, for training of on-device machine learning models 有权

    公开(公告)号：US12014739B2

    公开(公告)日：2024-06-18

    申请号：US18218818

    申请日：2023-07-06

    Applicant: GOOGLE LLC

    Inventor： Françoise Beaufays ,  Rajiv Mathews ,  Dragan Zivkovic ,  Kurt Partridge ,  Andrew Hard

    IPC: G10L15/22 ,  G10L15/065 ,  G10L15/10 ,  G10L15/30

    CPC classification number: G10L15/22 ,  G10L15/065 ,  G10L15/10 ,  G10L15/30

    Abstract: Processor(s) of a client device can: receive sensor data that captures environmental attributes of an environment of the client device; process the sensor data using a machine learning model to generate a predicted output that dictates whether one or more currently dormant automated assistant functions are activated; making a decision as to whether to trigger the one or more currently dormant automated assistant functions; subsequent to making the decision, determining that the decision was incorrect; and in response to determining that the determination was incorrect, generating a gradient based on comparing the predicted output to ground truth output. In some implementations, the generated gradient is used, by processor(s) of the client device, to update weights of the on-device speech recognition model. In some implementations, the generated gradient is additionally or alternatively transmitted to a remote system for use in remote updating of global weights of a global speech recognition model.

    Using corrections, of automated assistant functions, for training of on-device machine learning models
    4.
    发明授权
    Using corrections, of automated assistant functions, for training of on-device machine learning models 有权

    公开(公告)号：US11741953B2

    公开(公告)日：2023-08-29

    申请号：US16973572

    申请日：2019-11-08

    Applicant: Google LLC

    Inventor： Françoise Beaufays ,  Rajiv Mathews ,  Dragan Zivkovic ,  Kurt Partridge ,  Andrew Hard

    IPC: G10L15/22 ,  G10L15/065 ,  G10L15/10 ,  G10L15/30

    CPC classification number: G10L15/22 ,  G10L15/065 ,  G10L15/10 ,  G10L15/30

    Abstract: Processor(s) of a client device can: receive sensor data that captures environmental attributes of an environment of the client device; process the sensor data using a machine learning model to generate a predicted output that dictates whether one or more currently dormant automated assistant functions are activated; making a decision as to whether to trigger the one or more currently dormant automated assistant functions; subsequent to making the decision, determining that the decision was incorrect; and in response to determining that the determination was incorrect, generating a gradient based on comparing the predicted output to ground truth output. In some implementations, the generated gradient is used, by processor(s) of the client device, to update weights of the on-device speech recognition model. In some implementations, the generated gradient is additionally or alternatively transmitted to a remote system for use in remote updating of global weights of a global speech recognition model.

    SYSTEM(S) AND METHOD(S) FOR JOINTLY LEARNING MACHINE LEARNING MODEL(S) BASED ON SERVER DATA AND CLIENT DATA
    5.
    发明公开
    SYSTEM(S) AND METHOD(S) FOR JOINTLY LEARNING MACHINE LEARNING MODEL(S) BASED ON SERVER DATA AND CLIENT DATA 审中-公开

    公开(公告)号：US20230359907A1

    公开(公告)日：2023-11-09

    申请号：US17848947

    申请日：2022-07-01

    Applicant: GOOGLE LLC

    Inventor： Sean Augenstein ,  Andrew Hard ,  Kurt Partridge ,  Rajiv Mathews ,  Lin Ning ,  Karan Singhal

    IPC: G06N5/02

    CPC classification number: G06N5/022

    Abstract: Implementations disclosed herein are directed to various techniques for mitigating and/or preventing catastrophic forgetting in federated learning of global machine learning (ML) models. Implementations may identify a global ML model that is initially trained at a remote server based on a server data set, determine server-based data for global weight(s) of the global ML model, and transmit the global ML model and the server-based data to a plurality of client devices. The server-based data may include, for example, EWC loss term(s), client augmenting gradients, server augmenting gradients, and/or server-based data. Further, the plurality client devices may generate, based on processing corresponding predicted output and using the global ML model, and based on the server-based data, a corresponding client gradient, and transmit the corresponding client gradient to the remote server. Implementations may further generate an updated global ML model based on at least the corresponding client gradients.

    UTILIZING ELASTIC WEIGHT CONSOLIDATION (EWC) LOSS TERM(S) TO MITIGATE CATASTROPHIC FORGETTING IN FEDERATED LEARNING OF MACHINE LEARNING MODEL(S)
    6.
    发明公开
    UTILIZING ELASTIC WEIGHT CONSOLIDATION (EWC) LOSS TERM(S) TO MITIGATE CATASTROPHIC FORGETTING IN FEDERATED LEARNING OF MACHINE LEARNING MODEL(S) 审中-公开

    公开(公告)号：US20230351246A1

    公开(公告)日：2023-11-02

    申请号：US17734766

    申请日：2022-05-02

    Applicant: GOOGLE LLC

    Inventor： Andrew Hard ,  Kurt Partridge ,  Rajiv Mathews ,  Sean Augenstein

    IPC: G06N20/00 ,  H04L67/10

    CPC classification number: G06N20/00 ,  H04L67/10

    Abstract: Implementations disclosed herein are directed to utilizing elastic weight consolidation (EWC) loss term(s) in federated learning of global machine learning (ML) models. Implementations may identify a global ML model that initially trained at a remote server based on a server data set, determine the EWC loss term(s) for global weight(s) of the global ML model, and transmit the global ML model and the EWC loss term(s) to a plurality of client devices. The EWC loss term(s) may be determined based on a Fisher information matrix for the server data set. Further, the plurality client devices may generate, based on processing corresponding predicted output and using the global ML model, and based on the EWC loss term(s), a corresponding client gradient, and transmit the corresponding client gradient to the remote server. Implementations may further generate an updated global ML model based on at least the corresponding client gradients.

    USING CORRECTIONS, OF AUTOMATED ASSISTANT FUNCTIONS, FOR TRAINING OF ON-DEVICE MACHINE LEARNING MODELS
    7.
    发明公开
    USING CORRECTIONS, OF AUTOMATED ASSISTANT FUNCTIONS, FOR TRAINING OF ON-DEVICE MACHINE LEARNING MODELS 审中-公开

    公开(公告)号：US20240296843A1

    公开(公告)日：2024-09-05

    申请号：US18657405

    申请日：2024-05-07

    Applicant: GOOGLE LLC

    Inventor： Françoise Beaufays ,  Rajiv Mathews ,  Dragan Zivkovic ,  Kurt Partridge ,  Andrew Hard

    IPC: G10L15/22 ,  G10L15/065 ,  G10L15/10 ,  G10L15/30

    CPC classification number: G10L15/22 ,  G10L15/065 ,  G10L15/10 ,  G10L15/30

    Abstract: Processor(s) of a client device can: receive sensor data that captures environmental attributes of an environment of the client device; process the sensor data using a machine learning model to generate a predicted output that dictates whether one or more currently dormant automated assistant functions are activated; making a decision as to whether to trigger the one or more currently dormant automated assistant functions; subsequent to making the decision, determining that the decision was incorrect; and in response to determining that the determination was incorrect, generating a gradient based on comparing the predicted output to ground truth output. In some implementations, the generated gradient is used, by processor(s) of the client device, to update weights of the on-device speech recognition model. In some implementations, the generated gradient is additionally or alternatively transmitted to a remote system for use in remote updating of global weights of a global speech recognition model.

    USING CORRECTIONS, OF AUTOMATED ASSISTANT FUNCTIONS, FOR TRAINING OF ON-DEVICE MACHINE LEARNING MODELS
    8.
    发明申请
    USING CORRECTIONS, OF AUTOMATED ASSISTANT FUNCTIONS, FOR TRAINING OF ON-DEVICE MACHINE LEARNING MODELS 有权

    公开(公告)号：US20210327421A1

    公开(公告)日：2021-10-21

    申请号：US16973572

    申请日：2019-11-08

    Applicant: Google LLC

    Inventor： Françoise Beaufays ,  Rajiv Mathews ,  Dragan Zivkovic ,  Kurt Partridge ,  Andrew Hard

    IPC: G10L15/22 ,  G10L15/065 ,  G10L15/10 ,  G10L15/30

    Abstract: Processor(s) of a client device can: receive sensor data that captures environmental attributes of an environment of the client device; process the sensor data using a machine learning model to generate a predicted output that dictates whether one or more currently dormant automated assistant functions are activated; making a decision as to whether to trigger the one or more currently dormant automated assistant functions; subsequent to making the decision, determining that the decision was incorrect; and in response to determining that the determination was incorrect, generating a gradient based on comparing the predicted output to ground truth output. In some implementations, the generated gradient is used, by processor(s) of the client device, to update weights of the on-device speech recognition model. In some implementations, the generated gradient is additionally or alternatively transmitted to a remote system for use in remote updating of global weights of a global speech recognition model.

Patent Agency Ranking