公开(公告)号：US20230196211A1

公开(公告)日：2023-06-22

申请号：US18008293

申请日：2021-06-07

Applicant: Google LLC

Inventor： Carlos Riquelme Ruiz ,  André Susano Pinto ,  Joan Puigcerver ,  Basil Mustafa ,  Neil Matthew Tinmouth Houlsby ,  Sylvain Gelly ,  Cedric Benjamin Renggli ,  Daniel Martin Keysers

IPC: G06N20/20

CPC classification number: G06N20/20

Abstract: Generally, the present disclosure is directed to systems and methods that provide a simple, scalable, yet effective strategy to perform transfer learning with a mixture of experts (MoE). In particular, the transfer of pre-trained representations can improve sample efficiency and reduce computational requirements for new tasks. However, representations used for transfer are usually generic, and are not tailored to a particular distribution of downstream tasks. In contrast, example systems and methods of the present disclosure use expert representations for transfer with a simple, yet effective, strategy.