公开(公告)号：WO2022251719A1

公开(公告)日：2022-12-01

申请号：PCT/US2022/031468

申请日：2022-05-27

Applicant: GOOGLE LLC

Inventor： SO, David Richard ,  LE, Quoc V. ,  LIU, Hanxiao ,  MANKE, Wojciech Andrzej ,  DAI, Zihang ,  SHAZEER, Noam M.

IPC: G06N3/04 ,  G06N3/08 ,  G06N3/12

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining neural network architectures. One of the methods includes receiving initial neural network architecture data; generating, from the initial neural network architecture data, search space data defining a plurality of sub-model architectures, each sub-model architecture comprising an ordered set of primitive neural network operations each associated with one or more operation parameters; and determining a final architecture of a neural network for performing a machine learning task comprising running an evolutionary architecture search algorithm over the search space data to identify a respective optimized value for each of the one or more operation parameters of the primitive neural network operations in at least one of the plurality of sub-model architectures.

公开(公告)号：WO2022251602A1

公开(公告)日：2022-12-01

申请号：PCT/US2022/031304

申请日：2022-05-27

Applicant: GOOGLE LLC

Inventor： DAI, Zihang ,  LIU, Hanxiao ,  TAN, Mingxing ,  LE, Quoc V.

IPC: G06V10/44 ,  G06V10/82 ,  G06N3/04 ,  G06N3/08

Abstract: A computer-implemented method for performing computer vision with reduced computational cost and improved accuracy can include obtaining, by a computing system including one or more computing devices, input data comprising an input tensor having one or more dimensions, providing, by the computing system, the input data to a machine-learned convolutional attention network, the machine-learned convolutional attention network including two or more network stages, and, in response to providing the input data to the machine-learned convolutional attention network, receiving, by the computing system, a machine-learning prediction from the machine-learned convolutional attention network. The convolutional attention network can include at least one attention block, wherein the attention block includes a relative attention mechanism, the relative attention mechanism including the sum of a static convolution kernel with an adaptive attention matrix. This provides for improved generalization, capacity, and efficiency of the convolutional attention network relative to some existing models.

公开(公告)号：WO2023091511A2

公开(公告)日：2023-05-25

申请号：PCT/US2022/050143

申请日：2022-11-16

Applicant: GOOGLE LLC

Inventor： PHAM, Hieu Hy ,  DAI, Zihang ,  GHIASI, Golnaz ,  LIU, Hanxiao ,  YU, Wei ,  TAN, Mingxing ,  LE, Quoc V.

IPC: G06N3/0464 ,  G06N3/045 ,  G06N3/0442 ,  G06N3/084 ,  G06N3/09

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using memory-optimized contrastive learning to train image encoder and text encoder neural networks.

公开(公告)号：WO2022241320A1

公开(公告)日：2022-11-17

申请号：PCT/US2022/029470

申请日：2022-05-16

Applicant: GOOGLE LLC

Inventor： LIU, Hanxiao ,  SO, David Richard ,  LE, Quoc V. ,  DAI, Zihang

IPC: G06N3/04 ,  G06N3/08 ,  G06N3/0454 ,  G16H10/60 ,  G16H50/20

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a machine learning task on a network input to generate a network output. In one aspect, one of the systems includes a neural network configured to perform the machine learning task, the neural network including one or more blocks that each include a feedforward spatial transformation unit.