中科微点-全球专利检索

  1. Login
  2. Sign Up

Search Results

9 records (took 0.012 seconds)

    FLY PARAMETER COMPRESSION AND DECOMPRESSION TO FACILITATE FORWARD AND/OR BACK PROPAGATION AT CLIENTS DURING FEDERATED LEARNING
    2.
    发明申请
    FLY PARAMETER COMPRESSION AND DECOMPRESSION TO FACILITATE FORWARD AND/OR BACK PROPAGATION AT CLIENTS DURING FEDERATED LEARNING 有权

    公开(公告)号：US20240371362A1

    公开(公告)日：2024-11-07

    申请号：US18652587

    申请日：2024-05-01

    Applicant: GOOGLE LLC

    Inventor： Tien-Ju Yang ,  Yonghui Xiao ,  Giovanni Motta ,  Françoise Beaufays ,  Rajiv Mathews ,  Mingqing Chen

    IPC: G10L15/06

    Abstract: Implementations are directed to efficient federated learning of machine learning (ML) model(s) through on-the-fly decompression and compression of model parameters, of the ML model(s), when facilitating forward propagation and/or back propagation at client device(s). For example, implementations can transmit, from a remote system to a client device, a compressed on-device ML model that includes some compressed parameters. Further, the client device can, in performing forward propagation and/or back propagation using the on-device ML model, decompress those compressed parameters on-the-fly as the parameters are needed for the propagation. The propagation will utilize the decompressed parameters that were decompressed on the fly. Further, after the decompressed parameters are utilized, they can be deallocated from memory (while their compressed counterparts optionally remain in memory) to enable allocation of memory for further decompressed parameters that will be needed next and/or needed for other ongoing process(es).

    CHAIN OF THOUGHT REASONING FOR ASR
    5.
    发明申请
    CHAIN OF THOUGHT REASONING FOR ASR 有权

    公开(公告)号：US20250118293A1

    公开(公告)日：2025-04-10

    申请号：US18891615

    申请日：2024-09-20

    Applicant: Google LLC

    Inventor： Mingqing Chen ,  Rajiv Mathews ,  Andrew Hard ,  Swaroop Ramaswamy ,  Kilol Gupta

    IPC: G10L15/06 ,  G10L15/26 ,  G10L25/30

    Abstract: A method includes receiving a conversational training dataset including a plurality of conversational training samples, each training sample associated with a corresponding conversation and including: corresponding audio data characterizing a corresponding current utterance spoken by a user during a current turn in the corresponding conversation; a corresponding context for the corresponding current utterance including a transcript of a previous turn in the corresponding conversation that precedes the current turn; a corresponding ground-truth transcription of the corresponding current utterance; and a CoT annotation representing a corresponding logical relationship between the corresponding current utterance and the previous turn. The method also includes, for each corresponding conversational training sample in the conversational training dataset, training a speech model on the corresponding conversational training sample to teach the speech model to learn how to predict the corresponding logical relationship from the corresponding audio data and the corresponding context.

    SYSTEM(S) AND METHOD(S) TO REDUCE A TRANSFERABLE SIZE OF LANGUAGE MODEL(S) TO ENABLE DECENTRALIZED LEARNING THEREOF
    7.
    发明公开
    SYSTEM(S) AND METHOD(S) TO REDUCE A TRANSFERABLE SIZE OF LANGUAGE MODEL(S) TO ENABLE DECENTRALIZED LEARNING THEREOF 审中-公开

    公开(公告)号：US20240265269A1

    公开(公告)日：2024-08-08

    申请号：US18125613

    申请日：2023-03-23

    Applicant: GOOGLE LLC

    Inventor： Mingqing Chen ,  Lara McConnaughey ,  Kaan Ege Özgün ,  Rajiv Mathews ,  Françoise Beaufays

    IPC: G06N3/098 ,  G06F40/40 ,  G06N3/044

    CPC classification number: G06N3/098 ,  G06F40/40 ,  G06N3/044

    Abstract: Implementations disclosed herein are directed to techniques for enabling decentralized learning of global language models (LMs). Remote processor(s) of a remote system can obtain a global LM that includes a global embedding matrix, generate a global embedding mask for the global embedding matrix using a masking technique, apply the global embedding mask to global embedding matrix to generate a sparsified global LM that includes a masked global embedding matrix that is a masked version of the global embedding matrix, transmit the sparsified global LM to computing device(s) that are participating in a given round of decentralized learning for the global language model, receive corresponding updates from the computing device(s), and cause the global LM to be updated based on the corresponding updates. By generating the global embedding mask and applying it to the global embedding matrix, the transferable size of the global LM is reduced thereby enabling decentralized learning thereof.

    Federated Learning with Partially Trainable Networks
    9.
    发明公开
    Federated Learning with Partially Trainable Networks 审中-公开

    公开(公告)号：US20230214642A1

    公开(公告)日：2023-07-06

    申请号：US17568933

    申请日：2022-01-05

    Applicant: Google LLC

    Inventor： Hakim Sidahmed ,  Zheng Xu ,  Mingqing Chen ,  Yuan Cao ,  Ankush Garg

    IPC: G06N3/08

    CPC classification number: G06N3/08

    Abstract: Example aspects of the present disclosure provide a novel, resource-efficient approach for federated machine learning techniques with PTNs. The system can determine a first set of training parameters from a plurality of parameters of the global model. Additionally, the system can generate a random seed, using a random number generator, based on a set of frozen parameters. Moreover, the system can transmit, respectively to a plurality of client computing devices, a first set of training parameters and the random seed. Furthermore, the system can receive, respectively from the plurality of client computing devices, updates to one or more parameters in the first set of training parameters. Subsequently, the system can aggregate the updates to one or more parameters that are respectively received from the plurality of client computing devices. The system can modify one or more global parameters of the global model based on the aggregation.

Patent Agency Ranking