公开(公告)号：US20210182660A1

公开(公告)日：2021-06-17

申请号：US16716461

申请日：2019-12-16

Applicant: SoundHound, Inc.

Inventor： Asif Amirguliyev ,  Zili Li ,  Jonah Probell

IPC: G06N3/08 ,  G06N3/04 ,  H04L29/08 ,  H04L12/24 ,  G10L15/22 ,  G10L15/16 ,  G10L15/06

Abstract: Systems and methods for distributed training of a neural network model are described. Various embodiments include a master device and a slave device. The master device has a first version of the neural network model. The slave device is communicatively coupled to a first data source and the master device, and the first data source is inaccessible by the master device, in accordance with one embodiment. The slave device is remote from the master device. The master device is configured to output first configuration data for the neural network model based on the first version of the neural network model. The slave device is configured to use the first configuration data to instantiate a second version of the neural network model. The slave device is configured to train the second version of the neural network model using data from the first data source and to output second configuration data for the neural network model. The master device is configured to use the second configuration data to update parameters for the first version of the neural network model.

公开(公告)号：US11551083B2

公开(公告)日：2023-01-10

申请号：US16716497

申请日：2019-12-17

Applicant: SoundHound, Inc.

Inventor： Zili Li ,  Asif Amirguliyev ,  Jonah Probell

IPC: G06N3/08 ,  H04L67/10 ,  H04L41/082 ,  G06N3/04

Abstract: Training and enhancement of neural network models, such as from private data, are described. A slave device receives a version of a neural network model from a master. The slave accesses a local and/or private data source and uses the data to perform optimization of the neural network model. This can be done such as by computing gradients or performing knowledge distillation to locally train an enhanced second version of the model. The slave sends the gradients or enhanced neural network model to a master. The master may use the gradient or second version of the model to improve a master model.