公开(公告)号：US20250156715A1

公开(公告)日：2025-05-15

申请号：US18839379

申请日：2022-12-27

Applicant: Google LLC

Inventor： Da Huang ,  Chengrun Yang ,  Pieter-Jan Kindermans ,  Hanxiao Liu ,  Quoc V. Le ,  Madeleine Richards Udell ,  Yifeng Lu ,  Gabriel Mintzer Bender

IPC: G06N3/082

Abstract: Provided are neural architecture search techniques that have improved computational efficiency via performance of an initial constraint evaluation and improved gradient update approach. Further, the proposed approaches provide significant improvements for certain modalities of input data, such as tabular datasets.

公开(公告)号：US20250131251A1

公开(公告)日：2025-04-24

申请号：US18834070

申请日：2023-01-30

Applicant: Google LLC

Inventor： Hanxiao Liu ,  Quoc V. Le ,  Yanqi Zhou ,  Tao Lei ,  Yuzhe Zhao ,  Yanping Huang ,  Nan Du ,  Zhifeng Chen ,  Andrew M. Dai ,  James Laudon

IPC: G06N3/048

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a machine learning task on a network input to generate a network output. In one aspect, one of the systems includes a neural network configured to perform the machine learning task, the neural network including one or more expert neural network blocks that each include router that performs expert-choice routing between multiple expert neural networks.

公开(公告)号：US20240428071A1

公开(公告)日：2024-12-26

申请号：US18823611

申请日：2024-09-03

Applicant: Google LLC

Inventor： David Richard So ,  Quoc V. Le ,  Hanxiao Liu ,  Wojciech Andrzej Manke ,  Zihang Dai ,  Noam M. Shazeer

IPC: G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing a machine learning task on a network input to generate a network output. One of the systems includes an attention neural network configured to perform the machine learning task. The attention neural network includes one or more attentions layers that each include a squared ReLU activation layer, a depth-wise convolution layer, or both.

公开(公告)号：US11755883B2

公开(公告)日：2023-09-12

申请号：US17827130

申请日：2022-05-27

Applicant: Google LLC

Inventor： Zihang Dai ,  Hanxiao Liu ,  Mingxing Tan ,  Quoc V. Le

IPC: G06N20/00 ,  G06N3/044 ,  G06N3/063 ,  G06N3/08

CPC classification number: G06N3/044 ,  G06N3/063 ,  G06N3/08 ,  G06N20/00

Abstract: A computer-implemented method for performing computer vision with reduced computational cost and improved accuracy can include obtaining, by a computing system including one or more computing devices, input data comprising an input tensor having one or more dimensions, providing, by the computing system, the input data to a machine-learned convolutional attention network, the machine-learned convolutional attention network including two or more network stages, and, in response to providing the input data to the machine-learned convolutional attention network, receiving, by the computing system, a machine-learning prediction from the machine-learned convolutional attention network. The convolutional attention network can include at least one attention block, wherein the attention block includes a relative attention mechanism, the relative attention mechanism including the sum of a static convolution kernel with an adaptive attention matrix. This provides for improved generalization, capacity, and efficiency of the convolutional attention network relative to some existing models.

公开(公告)号：US20220405579A1

公开(公告)日：2022-12-22

申请号：US17613773

申请日：2021-03-03

Applicant: Google LLC

Inventor： Jiahui Yu ,  Pengchong Jin ,  Hanxiao Liu ,  Gabriel Mintzer Bender ,  Pieter-Jan Kindermans ,  Mingxing Tan ,  Xiaodan Song ,  Ruoming Pang ,  Quoc V. Le

IPC: G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for selecting a neural network to perform a particular machine learning task while satisfying a set of constraints.