公开(公告)号：US20250110940A1

公开(公告)日：2025-04-03

申请号：US18905090

申请日：2024-10-02

Applicant: Google LLC

Inventor： Xin Yang Yak ,  Sercan Omer Arik ,  Yihe Dong ,  Javier Gonzalvo Fructuoso

IPC: G06F16/22 ,  G06N3/08

Abstract: Methods, systems, and apparatuses, including computer programs encoded on computer storage media, for implementing a neural network that can perform one or more machine learning tasks on an input that includes data that represents a given data structure. In particular, implementing a language model to encode the data and a foundation neural network with an attention-based architecture to generate the task output. Because of how language model generated embeddings are defined and cached, the described techniques demonstrate significant improvements in required computational resources for training and inference while also exceeding prediction performance on a variety of prediction tasks over conventional approaches.