公开(公告)号：US20240412042A1

公开(公告)日：2024-12-12

申请号：US18698260

申请日：2022-10-06

Applicant: DeepMind Technologies Limited

Inventor： Nikolay Savinov ,  Junyoung Chung ,  Mikolaj Binkowski ,  Aaron Gerard Antonius van den Oord ,  Erich Konrad Elsen

IPC: G06N3/0455 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating output sequences using a non-auto-regressive neural network.

公开(公告)号：US20240119261A1

公开(公告)日：2024-04-11

申请号：US18374447

申请日：2023-09-28

Applicant: DeepMind Technologies Limited

Inventor： Robin Strudel ,  Rémi Leblond ,  Laurent Sifre ,  Sander Etienne Lea Dieleman ,  Nikolay Savinov ,  Will S. Grathwohl ,  Corentin Tallec ,  Florent Altché ,  Iaroslav Ganin ,  Arthur Mensch ,  Yilin Du

IPC: G06N3/045

CPC classification number: G06N3/045

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an output sequence of discrete tokens using a diffusion model. In one aspect, a method includes generating, by using the diffusion model, a final latent representation of the sequence of discrete tokens that includes a determined value for each of a plurality of latent variables; applying a de-embedding matrix to the final latent representation of the output sequence of discrete tokens to generate a de-embedded final latent representation that includes, for each of the plurality of latent variables, a respective numeric score for each discrete token in a vocabulary of multiple discrete tokens; selecting, for each of the plurality of latent variables, a discrete token from among the multiple discrete tokens in the vocabulary that has a highest numeric score; and generating the output sequence of discrete tokens that includes the selected discrete tokens.