公开(公告)号：US11651218B1

公开(公告)日：2023-05-16

申请号：US17888230

申请日：2022-08-15

Applicant: Google LLC

Inventor： Christian Szegedy ,  Ian Goodfellow

IPC: G06N3/08 ,  G06N20/00

CPC classification number: G06N3/08 ,  G06N20/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for adversarial training of a neural network. One of the methods includes obtaining a plurality of training inputs; and training the neural network on each of the training inputs, comprising, for each of the training inputs: processing the training input using the neural network to determine a neural network output for the training input; applying a perturbation to the training input to generate an adversarial perturbation of the training input; processing the adversarial perturbation of the training input using the neural network to determine a neural network output for the adversarial perturbation; and adjusting the current values of the parameters of the neural network by performing an iteration of a neural network training procedure to optimize an adversarial objective function.

公开(公告)号：US11416745B1

公开(公告)日：2022-08-16

申请号：US16692257

申请日：2019-11-22

Applicant: Google LLC

Inventor： Christian Szegedy ,  Ian Goodfellow

IPC: G06N3/08 ,  G06N20/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for adversarial training of a neural network. One of the methods includes obtaining a plurality of training inputs; and training the neural network on each of the training inputs, comprising, for each of the training inputs: processing the training input using the neural network to determine a neural network output for the training input; applying a perturbation to the training input to generate an adversarial perturbation of the training input; processing the adversarial perturbation of the training input using the neural network to determine a neural network output for the adversarial perturbation; and adjusting the current values of the parameters of the neural network by performing an iteration of a neural network training procedure to optimize an adversarial objective function.

公开(公告)号：US10699191B2

公开(公告)日：2020-06-30

申请号：US15349901

申请日：2016-11-11

Applicant: Google LLC

Inventor： Ian Goodfellow ,  Tianqi Chen ,  Jonathan Shlens

IPC: G06N3/08 ,  G06N3/04

Abstract: This specification describes methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a larger neural network from a smaller neural network. One of the described methods includes obtaining data specifying an original neural network and generating a larger neural network from the original neural network The larger neural network has a larger neural network structure than the original neural network structure. The values of the parameters of the original neural network units and the additional neural network units are initialized so that the larger neural network generates the same outputs from the same inputs as the original neural network and the larger neural network is trained to determine trained values of the parameters of the original neural network units and the additional neural network units from the initialized values.

公开(公告)号：US11790233B2

公开(公告)日：2023-10-17

申请号：US16915502

申请日：2020-06-29

Applicant: Google LLC

Inventor： Ian Goodfellow ,  Tianqi Chen ,  Jonathon Shlens

IPC: G06N3/08 ,  G06N3/04 ,  G06N3/082 ,  G06N3/045

CPC classification number: G06N3/082 ,  G06N3/04 ,  G06N3/045

Abstract: The specification describes methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a larger neural network from a smaller neural network. One of the described methods includes obtaining data specifying an original neural network and generating a larger neural network from the original neural network. The larger neural network has a larger neural network structure than the original neural network structure. The values of the parameters of the original neural network units and the additional neural network units are initialized so that the larger neural network generates the same outputs from the same inputs as the original neural network, and the larger neural network is trained to determine trained values of the parameters of the original neural network units and the additional neural network units from the initialized values.

公开(公告)号：US20220063089A1

公开(公告)日：2022-03-03

申请号：US17524185

申请日：2021-11-11

Applicant: GOOGLE LLC

Inventor： Sergey Levine ,  Chelsea Finn ,  Ian Goodfellow

IPC: B25J9/16 ,  G05B13/02 ,  G06N3/08 ,  G06K9/46 ,  G06K9/00 ,  G06N3/00 ,  G06N3/04 ,  G06T7/73

Abstract: Some implementations of this specification are directed generally to deep machine learning methods and apparatus related to predicting motion(s) (if any) that will occur to object(s) in an environment of a robot in response to particular movement of the robot in the environment. Some implementations are directed to training a deep neural network model to predict at least one transformation (if any), of an image of a robot's environment, that will occur as a result of implementing at least a portion of a particular movement of the robot in the environment. The trained deep neural network model may predict the transformation based on input that includes the image and a group of robot movement parameters that define the portion of the particular movement.

公开(公告)号：US11869170B2

公开(公告)日：2024-01-09

申请号：US17293754

申请日：2019-11-18

Applicant: Google LLC

Inventor： David Berthelot ,  Ian Goodfellow

IPC: G06T3/40 ,  G06N3/08 ,  G06T5/50 ,  G06F18/22 ,  G06N3/045

CPC classification number: G06T3/4046 ,  G06F18/22 ,  G06N3/045 ,  G06N3/08 ,  G06T3/4053 ,  G06T5/50

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network. One of the methods includes receiving a training image and a ground truth super-resolution image; processing a first training network input comprising the training image using the neural network to generate a first training super-resolution image; processing a first critic input generated from (i) the training image and (ii) the ground truth super-resolution image using a critic neural network to map the first critic input to a latent representation; processing a second critic input generated from (i) the training image and (ii) the first training super-resolution image using the critic neural network to map the second critic input to a latent representation; determining a gradient of a generator loss function that measures a distance between the latent representations of the critic inputs; and determining an update to the parameters.

公开(公告)号：US11354574B2

公开(公告)日：2022-06-07

申请号：US16859789

申请日：2020-04-27

Applicant: Google LLC

Inventor： Aurko Roy ,  Ian Goodfellow ,  Jacob Buckman ,  Colin Abraham Raffel

IPC: G06N3/08 ,  G06F17/16 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for increasing the security of neural network by discretizing neural network inputs. One of the methods includes receiving a network input for a neural network; processing the network input using a discretization layer, wherein the discretization layer is configured to generate a discretized network input comprising a respective discretized vector for each of the numeric values in the network input; and processing the discretized network input using the plurality of additional neural network layers to generate a network output for the network input.

公开(公告)号：US11514313B2

公开(公告)日：2022-11-29

申请号：US16580649

申请日：2019-09-24

Applicant: Google LLC

Inventor： Samaneh Azadi ,  Ian Goodfellow ,  Catherine Olsson ,  Augustus Quadrozzi Odena

IPC: G06N3/08 ,  G06N3/04 ,  G10L15/16

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for providing a data sample in response to a request for a data sample. In one aspect, a method comprises: receiving a request for a new data sample; until a candidate new data sample is generated that satisfies an acceptance criterion, performing operations comprising: generating a candidate new data sample using a generator neural network; processing the candidate new data sample using a discriminator neural network to generate an imitation score; and determining, from the imitation score, whether the candidate new data sample satisfies the acceptance criterion; and providing the candidate new data sample that satisfies the acceptance criterion in response to the received request.

公开(公告)号：US20200257978A1

公开(公告)日：2020-08-13

申请号：US16859789

申请日：2020-04-27

Applicant: Google LLC

Inventor： Aurko Roy ,  Ian Goodfellow ,  Jacob Buckman ,  Colin Abraham Raffel

IPC: G06N3/08 ,  G06N3/04 ,  G06F17/16

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for increasing the security of neural network by discretizing neural network inputs. One of the methods includes receiving a network input for a neural network; processing the network input using a discretization layer, wherein the discretization layer is configured to generate a discretized network input comprising a respective discretized vector for each of the numeric values in the network input; and processing the discretized network input using the plurality of additional neural network layers to generate a network output for the network input.