公开(公告)号：US20230117768A1

公开(公告)日：2023-04-20

申请号：US17965274

申请日：2022-10-13

申请人： Kiarash SHALOUDEGI ,  Yaoliang YU ,  Jun LUO

发明人： Kiarash SHALOUDEGI ,  Yaoliang YU ,  Jun LUO

IPC分类号： G06N20/00 ,  G06K9/62

摘要： Methods and systems for federated learning using a parameterized optimization algorithm are described. A central server receives, from each of a plurality of user devices, a proximal map and feedback representing a current state of each user device. The server computes an update to optimization parameters of a parameterized optimization algorithm, using the received feedback. Model updates are computed for each user device, using the received proximal maps and the parameterized optimization algorithm having the updated optimization parameters. Each model update is transmitted to each respective client for updating the respective model.

公开(公告)号：US20210365841A1

公开(公告)日：2021-11-25

申请号：US16881999

申请日：2020-05-22

申请人： Kiarash SHALOUDEGI ,  Yaoliang YU

发明人： Kiarash SHALOUDEGI ,  Yaoliang YU

IPC分类号： G06N20/20

摘要： Methods and apparatuses for implementing federated learning are described. A set of updates is obtained, where each update represents a respective difference between a global model and a respective local model. The global model is updated using a weighted average of the set of updates. A set of weighting coefficients is calculated, to be used in calculating the weighted average. The set of weighting coefficients is calculated by performing multi-objective optimization towards a Pareto-stationary solution across the set of updates. The weighted average is calculated by applying the set of weighting coefficients to the set of updates, and the global model is updated by adding the weighted average to the global model.

公开(公告)号：US20220237508A1

公开(公告)日：2022-07-28

申请号：US17161224

申请日：2021-01-28

申请人： Kiarash SHALOUDEGI ,  Rasul TUTUNOV ,  Haitham BOU AMMAR

发明人： Kiarash SHALOUDEGI ,  Rasul TUTUNOV ,  Haitham BOU AMMAR

IPC分类号： G06N20/00 ,  G06K9/62 ,  G06F17/16 ,  G06F17/15

摘要： Servers, methods and systems for second order federated learning (FL) are described. Client nodes send local curvature information to the server along with local learned parameter information. The local curvature information enables the server to approximate or estimate the curvature, i.e. a second-order derivative, of an objective function of each respective local model. Instead of averaging the local learned parameter information obtained from the client nodes, the server uses the local curvature information to aggregate the local learned parameter information obtained from each client node to correct for the bias that would ordinarily result from a straightforward averaging of the learned values of the local learnable parameters. The described examples may provide reduced bias and/or reduced communication costs, relative to existing FL approaches such as federated averaging. The described examples may provide greater accuracy in model performance and/or faster convergence in FL.

公开(公告)号：US20210383280A1

公开(公告)日：2021-12-09

申请号：US16891752

申请日：2020-06-03

申请人： Kiarash SHALOUDEGI

发明人： Kiarash SHALOUDEGI

IPC分类号： G06N20/20 ,  G06F17/16

摘要： Methods and computing apparatuses for defending against model poisoning attacks in federated learning are described. One or more updates are obtained, where each update represents a respective difference between parameters (e.g. weights) of the global model and parameters (e.g. weights) of a respective local model. Random noise perturbation and normalization are applied to each update, to obtain one or more perturbed and normalized updates. The parameters (e.g. weights) of the global model are updated by adding an aggregation of the one or more perturbed and normalized updates to the parameters (e.g. weights) of the global model. In some examples, one or more learned parameters (e.g. weights) of the previous global model are also perturbed using random noise.