公开(公告)号：US12282857B1

公开(公告)日：2025-04-22

申请号：US18900506

申请日：2024-09-27

Applicant: Google LLC

Inventor： Siyuan Qiao ,  Chenxi Liu ,  Jiahui Yu ,  Yonghui Wu

IPC: G06N3/088

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training neural networks through contrastive learning. In particular, the contrastive learning is modified to use a relative margin to adjust a training pair's contribution to optimization.

公开(公告)号：US20210334624A1

公开(公告)日：2021-10-28

申请号：US17365939

申请日：2021-07-01

Applicant: Google LLC

Inventor： Wei Hua ,  Barret Zoph ,  Jonathon Shlens ,  Chenxi Liu ,  Jonathan Huang ,  Jia Li ,  Fei-Fei Li ,  Kevin Patrick Murphy

IPC: G06N3/04 ,  G06K9/62 ,  G06N3/063 ,  G06N3/08

Abstract: A method for determining an architecture for a task neural network configured to perform a particular machine learning task is described. The method includes obtaining data specifying a current set of candidate architectures for the task neural network; for each candidate architecture in the current set: processing the data specifying the candidate architecture using a performance prediction neural network having multiple performance prediction parameters, the performance prediction neural network being configured to process the data specifying the candidate architecture in accordance with current values of the performance prediction parameters to generate a performance prediction that characterizes how well a neural network having the candidate architecture would perform after being trained on the particular machine learning task; and generating an updated set of candidate architectures by selecting one or more of the candidate architectures in the current set based on the performance predictions for the candidate architectures in the current set.

公开(公告)号：US20200257961A1

公开(公告)日：2020-08-13

申请号：US16861491

申请日：2020-04-29

Applicant: Google LLC

Inventor： Wei Hua ,  Barret Zoph ,  Jonathon Shlens ,  Chenxi Liu ,  Jonathan Huang ,  Jia Li ,  Fei-Fei Li ,  Kevin Patrick Murphy

IPC: G06N3/04 ,  G06N3/063 ,  G06N3/08 ,  G06K9/62

Abstract: A method for determining an architecture for a task neural network configured to perform a particular machine learning task is described. The method includes obtaining data specifying a current set of candidate architectures for the task neural network; for each candidate architecture in the current set: processing the data specifying the candidate architecture using a performance prediction neural network having multiple performance prediction parameters, the performance prediction neural network being configured to process the data specifying the candidate architecture in accordance with current values of the performance prediction parameters to generate a performance prediction that characterizes how well a neural network having the candidate architecture would perform after being trained on the particular machine learning task; and generating an updated set of candidate architectures by selecting one or more of the candidate architectures in the current set based on the performance predictions for the candidate architectures in the current set.

公开(公告)号：US11087201B2

公开(公告)日：2021-08-10

申请号：US16861491

申请日：2020-04-29

Applicant: Google LLC

Inventor： Wei Hua ,  Barret Zoph ,  Jonathon Shlens ,  Chenxi Liu ,  Jonathan Huang ,  Jia Li ,  Fei-Fei Li ,  Kevin Patrick Murphy

IPC: G06N3/04 ,  G06K9/62 ,  G06N3/063 ,  G06N3/08

Abstract: A method for determining an architecture for a task neural network configured to perform a particular machine learning task is described. The method includes obtaining data specifying a current set of candidate architectures for the task neural network; for each candidate architecture in the current set: processing the data specifying the candidate architecture using a performance prediction neural network having multiple performance prediction parameters, the performance prediction neural network being configured to process the data specifying the candidate architecture in accordance with current values of the performance prediction parameters to generate a performance prediction that characterizes how well a neural network having the candidate architecture would perform after being trained on the particular machine learning task; and generating an updated set of candidate architectures by selecting one or more of the candidate architectures in the current set based on the performance predictions for the candidate architectures in the current set.

公开(公告)号：US20250111235A1

公开(公告)日：2025-04-03

申请号：US18900506

申请日：2024-09-27

Applicant: Google LLC

Inventor： Siyuan Qiao ,  Chenxi Liu ,  Jiahui Yu ,  Yonghui Wu

IPC: G06N3/088

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training neural networks through contrastive learning. In particular, the contrastive learning is modified to use a relative margin to adjust a training pair's contribution to optimization.