公开(公告)号：US20240005129A1

公开(公告)日：2024-01-04

申请号：US18029849

申请日：2021-10-01

Applicant: Google LLC

Inventor： Yanqi Zhou ,  Amir Yazdanbakhsh ,  Berkin Akin ,  Daiyi Peng ,  Yuxiong Zhu ,  Mingxing Tan ,  Xuanyi Dong

IPC: G06N3/045 ,  G06N3/092 ,  G06N3/063 ,  G06N3/044 ,  G06N3/0464

CPC classification number: G06N3/045 ,  G06N3/092 ,  G06N3/0464 ,  G06N3/044 ,  G06N3/063

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for jointly determining neural network architectures and hardware accelerator architectures. In one aspect, a method includes: generating, using a controller policy, a batch of one or more output sequences, each output sequence in the batch defining a respective architecture of a child neural network and a respective architecture of a hardware accelerator; for each output sequence in the batch: training a respective instance of the child neural network having the architecture defined by the output sequence; evaluating a network performance of the trained instance of the child neural; and evaluating an accelerator performance of a respective instance of the hardware accelerator having the architecture defined by the output sequence to determine an accelerator performance metric for the instance of the hardware accelerator; and using the network performance metrics and the accelerator performance metrics to adjust the controller policy.

公开(公告)号：US20210383223A1

公开(公告)日：2021-12-09

申请号：US17337834

申请日：2021-06-03

Applicant: Google LLC

Inventor： Mingxing Tan ,  Xuanyi Dong ,  Wei Yu ,  Quoc V. Le ,  Daiyi Peng

IPC: G06N3/08 ,  G06N3/04

Abstract: The present disclosure provides a differentiable joint hyper-parameter and architecture search approach, with some implementations including the idea of discretizing the continuous space into a linear combination of multiple categorical basis. One example element of the proposed approach is the use of weight sharing across all architecture- and hyper-parameters which enables it to search efficiently over the large joint search space. Experimental results on MobileNet/ResNet/EfficientNet/BERT show that the proposed systems significantly improve the accuracy by up to 2% on ImageNet and the F1 by up to 0.4 on SQuAD, with search cost comparable to training a single model. Compared to other AutoML methods, such as random search or Bayesian method, the proposed techniques can achieve better accuracy with 10× less compute cost.