公开(公告)号：US20240378427A1

公开(公告)日：2024-11-14

申请号：US18661499

申请日：2024-05-10

Applicant: Google LLC

Inventor： Slav Petrov ,  Yonghui Wu ,  Andrew M. Dai ,  David Richard So ,  Dmitry Lepikhin ,  Erica Ann Moreira ,  Gaurav Mishra ,  Jonathan Hudson Clark ,  Maxim Krikun ,  Melvin Jose Johnson Premkumar ,  Nan Du ,  Orhan Firat ,  Rohan Anil ,  Siamak Shakeri ,  Xavier Garcia ,  Yanping Huang ,  Yong Cheng ,  Yuanzhong Xu ,  Yujing Zhang ,  Zachary Alexander Nado ,  Eric Jun Jie Ni ,  Kefan Xiao ,  Vladimir Feinberg ,  Jin Young Sohn ,  Aurko Roy

IPC: G06N3/0475 ,  G06F40/284

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to perform any one or more of a variety of machine learning tasks. For example, the neural network can be configured as a generative neural network, e.g., an autoregressive generative neural network.

公开(公告)号：US20200257978A1

公开(公告)日：2020-08-13

申请号：US16859789

申请日：2020-04-27

Applicant: Google LLC

Inventor： Aurko Roy ,  Ian Goodfellow ,  Jacob Buckman ,  Colin Abraham Raffel

IPC: G06N3/08 ,  G06N3/04 ,  G06F17/16

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for increasing the security of neural network by discretizing neural network inputs. One of the methods includes receiving a network input for a neural network; processing the network input using a discretization layer, wherein the discretization layer is configured to generate a discretized network input comprising a respective discretized vector for each of the numeric values in the network input; and processing the discretized network input using the plurality of additional neural network layers to generate a network output for the network input.

公开(公告)号：US20240378441A1

公开(公告)日：2024-11-14

申请号：US18661447

申请日：2024-05-10

Applicant: Google LLC

Inventor： Slav Petrov ,  Yonghui Wu ,  Andrew M. Dai ,  David Richard So ,  Dmitry Lepikhin ,  Erica Ann Moreira ,  Gaurav Mishra ,  Jonathan Hudson Clark ,  Maxim Krikun ,  Melvin Jose Johnson Premkumar ,  Nan Du ,  Orhan Firat ,  Rohan Anil ,  Siamak Shakeri ,  Xavier Garcia ,  Yanping Huang ,  Yong Cheng ,  Yuanzhong Xu ,  Yujing Zhang ,  Zachary Alexander Nado ,  Eric Jun Jie Ni ,  Kefan Xiao ,  Vladimir Feinberg ,  Jin Young Sohn ,  Aurko Roy

IPC: G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to perform any one or more of a variety of machine learning tasks. For example, the neural network can be configured as a generative neural network, e.g., an autoregressive generative neural network.

公开(公告)号：US20240078379A1

公开(公告)日：2024-03-07

申请号：US17903805

申请日：2022-09-06

Applicant: Google LLC

Inventor： Rohan Anil ,  Aurko Roy

IPC: G06F40/20 ,  G06N3/04

CPC classification number: G06F40/20 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a machine learning task on a network input to generate a network output. In one aspect, one of the systems includes a neural network configured to perform the machine learning task, the neural network comprising an N-grammer layer and an output neural network, the N-grammer layer configured to: at each of one or more heads: receive a sequence of input embeddings; generate a discrete latent representation of the sequence of input embeddings by using a learned product quantization codebook; generate a plurality of n-gram indices from the discrete latent representation; and generate a latent n-gram representation of the sequence of input embeddings; and generate a sequence of output embeddings, and the output neural network configured to: receive the sequence of output embeddings; and process the sequence of output embeddings to generate the network output.

公开(公告)号：US11354574B2

公开(公告)日：2022-06-07

申请号：US16859789

申请日：2020-04-27

Applicant: Google LLC

Inventor： Aurko Roy ,  Ian Goodfellow ,  Jacob Buckman ,  Colin Abraham Raffel

IPC: G06N3/08 ,  G06F17/16 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for increasing the security of neural network by discretizing neural network inputs. One of the methods includes receiving a network input for a neural network; processing the network input using a discretization layer, wherein the discretization layer is configured to generate a discretized network input comprising a respective discretized vector for each of the numeric values in the network input; and processing the discretized network input using the plurality of additional neural network layers to generate a network output for the network input.