公开(公告)号：US20220398500A1

公开(公告)日：2022-12-15

申请号：US17332893

申请日：2021-05-27

Applicant: Google LLC

Inventor： Karan Singhal ,  Hakim Sidahmed, JR. ,  Zachary A. Garrett ,  Shanshan Wu ,  John Keith Rush ,  Sushant Prakash

IPC: G06N20/20

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a machine learning model having a set of local model parameters and a set of global model parameters under a partially local federated learning framework. One of the methods include maintaining local data and data defining the local model parameters; receiving data defining current values of the global model parameters; determining, based on the local data, the local model parameters, and the current values of the global model parameters, current values of the local model parameters; determining, based on the local data, the current values of the local model parameters, and the current values of the global model parameters, updated values of the global model parameters; generating, based on the updated values of the global model parameters, parameter update data defining an update to the global model parameters; and transmitting the parameter update data.

公开(公告)号：US20230359907A1

公开(公告)日：2023-11-09

申请号：US17848947

申请日：2022-07-01

Applicant: GOOGLE LLC

Inventor： Sean Augenstein ,  Andrew Hard ,  Kurt Partridge ,  Rajiv Mathews ,  Lin Ning ,  Karan Singhal

IPC: G06N5/02

CPC classification number: G06N5/022

Abstract: Implementations disclosed herein are directed to various techniques for mitigating and/or preventing catastrophic forgetting in federated learning of global machine learning (ML) models. Implementations may identify a global ML model that is initially trained at a remote server based on a server data set, determine server-based data for global weight(s) of the global ML model, and transmit the global ML model and the server-based data to a plurality of client devices. The server-based data may include, for example, EWC loss term(s), client augmenting gradients, server augmenting gradients, and/or server-based data. Further, the plurality client devices may generate, based on processing corresponding predicted output and using the global ML model, and based on the server-based data, a corresponding client gradient, and transmit the corresponding client gradient to the remote server. Implementations may further generate an updated global ML model based on at least the corresponding client gradients.

公开(公告)号：US20240428937A1

公开(公告)日：2024-12-26

申请号：US18338003

申请日：2023-06-20

Applicant: Google LLC

Inventor： Vivek Natarajan ,  Karan Singhal ,  Shekoofeh Azizi ,  Alan Prasana Karthikesalingam ,  Tao Tu ,  Seyedeh Sara Mahdavi ,  Christopher Semturs

IPC: G16H50/20

Abstract: An aspect of the present disclosure provides an example method comprising: receiving an input query associated with a particular task domain of a plurality of available task domains; obtaining a machine-learned prompt component and a curated prompt component, wherein the machine-learned prompt component comprises a plurality of machine-learned prompt values for the plurality of available task domains, and wherein the curated prompt component comprises a plurality of exemplar prompt values corresponding to one or more embedded natural language exemplars for the particular task domain from domain experts; and generating an output responsive to the input query by processing a combined prompt and the input query using a pre-trained machine-learned model, wherein the combined prompt comprises the machine-learned prompt component and the curated prompt component.