公开(公告)号：US20210110115A1

公开(公告)日：2021-04-15

申请号：US16497602

申请日：2018-06-05

Applicant: DeepMind Technologies Limited

Inventor： Karl Moritz Hermann ,  Philip Blunsom ,  Felix George Hill

IPC: G06F40/30 ,  G06N3/08 ,  G06F17/16

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent interacting with an environment. In one aspect, a system includes a language encoder model that is configured to receive a text string in a particular natural language, and process the text string to generate a text embedding of the text string. The system includes an observation encoder neural network that is configured to receive an observation characterizing a state of the environment, and process the observation to generate an observation embedding of the observation. The system includes a subsystem that is configured to obtain a current text embedding of a current text string and a current observation embedding of a current observation. The subsystem is configured to select an action to be performed by the agent in response to the current observation.

公开(公告)号：US20220318516A1

公开(公告)日：2022-10-06

申请号：US17744921

申请日：2022-05-16

Applicant: DeepMind Technologies Limited

Inventor： Karl Moritz Hermann ,  Philip Blunsom ,  Felix George Hill

IPC: G06F40/30 ,  G06F17/16 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent interacting with an environment. In one aspect, a system includes a language encoder model that is configured to receive a text string in a particular natural language, and process the text string to generate a text embedding of the text string. The system includes an observation encoder neural network that is configured to receive an observation characterizing a state of the environment, and process the observation to generate an observation embedding of the observation. The system includes a subsystem that is configured to obtain a current text embedding of a current text string and a current observation embedding of a current observation. The subsystem is configured to select an action to be performed by the agent in response to the current observation.

公开(公告)号：US11354509B2

公开(公告)日：2022-06-07

申请号：US16497602

申请日：2018-06-05

Applicant: DeepMind Technologies Limited

Inventor： Karl Moritz Hermann ,  Philip Blunsom ,  Felix George Hill

IPC: G06F40/30 ,  G06F17/16 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent interacting with an environment. In one aspect, a system includes a language encoder model that is configured to receive a text string in a particular natural language, and process the text string to generate a text embedding of the text string. The system includes an observation encoder neural network that is configured to receive an observation characterizing a state of the environment, and process the observation to generate an observation embedding of the observation. The system includes a subsystem that is configured to obtain a current text embedding of a current text string and a current observation embedding of a current observation. The subsystem is configured to select an action to be performed by the agent in response to the current observation.

公开(公告)号：US20240020972A1

公开(公告)日：2024-01-18

申请号：US18029980

申请日：2021-10-01

Applicant: DeepMind Technologies Limited

Inventor： Fengning Ding ,  Adam Anthony Santoro ,  Felix George Hill ,  Matthew Botvinick ,  Luis Piloto

IPC: G06V20/40 ,  G06V10/26 ,  G06V10/82 ,  G06V10/776

CPC classification number: G06V20/41 ,  G06V10/26 ,  G06V10/82 ,  G06V10/776

Abstract: A video processing system configured to analyze a sequence of video frames to detect objects in the video frames and provide information relating to the detected objects in response to a query. The query may comprise, for example, a request for a prediction of a future event, or of the location of an object, or a request for a prediction of what would happen if an object were modified. The system uses a transformer neural network subsystem to process representations of objects in the video.

公开(公告)号：US20230401835A1

公开(公告)日：2023-12-14

申请号：US18199896

申请日：2023-05-19

Applicant: DeepMind Technologies Limited

Inventor： Aaditya K. Singh ,  Fengning Ding ,  Felix George Hill ,  Andrew Kyle Lampinen

IPC: G06V10/82 ,  G06V20/62

CPC classification number: G06V10/82 ,  G06V20/635

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a speaker neural network using one or more listener neural networks.

公开(公告)号：US12265795B2

公开(公告)日：2025-04-01

申请号：US18649774

申请日：2024-04-29

Applicant: DeepMind Technologies Limited

Inventor： Karl Moritz Hermann ,  Philip Blunsom ,  Felix George Hill

IPC: G06F40/30 ,  G06F17/16 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent interacting with an environment. In one aspect, a system includes a language encoder model that is configured to receive a text string in a particular natural language, and process the text string to generate a text embedding of the text string. The system includes an observation encoder neural network that is configured to receive an observation characterizing a state of the environment, and process the observation to generate an observation embedding of the observation. The system includes a subsystem that is configured to obtain a current text embedding of a current text string and a current observation embedding of a current observation. The subsystem is configured to select an action to be performed by the agent in response to the current observation.

公开(公告)号：US20240320438A1

公开(公告)日：2024-09-26

申请号：US18649774

申请日：2024-04-29

Applicant: DeepMind Technologies Limited

Inventor： Karl Moritz Hermann ,  Philip Blunsom ,  Felix George Hill

IPC: G06F40/30 ,  G06F17/16 ,  G06N3/08

CPC classification number: G06F40/30 ,  G06F17/16 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent interacting with an environment. In one aspect, a system includes a language encoder model that is configured to receive a text string in a particular natural language, and process the text string to generate a text embedding of the text string. The system includes an observation encoder neural network that is configured to receive an observation characterizing a state of the environment, and process the observation to generate an observation embedding of the observation. The system includes a subsystem that is configured to obtain a current text embedding of a current text string and a current observation embedding of a current observation. The subsystem is configured to select an action to be performed by the agent in response to the current observation.

公开(公告)号：US20240282094A1

公开(公告)日：2024-08-22

申请号：US18568561

申请日：2022-06-08

Applicant: DeepMind Technologies Limited

Inventor： Maria Rafailia Tsimpoukelli ,  Jacob Lee Menick ,  Serkan Cabi ,  Felix George Hill ,  Seyed Mohammadali Eslami ,  Oriol Vinyals

IPC: G06V10/82 ,  G06F40/284 ,  G06V20/70

CPC classification number: G06V10/82 ,  G06F40/284 ,  G06V20/70

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing multi-modal inputs using language models. In particular, the inputs include an image, and the image is encoded by an image encoder neural network to generate a sequence of image embeddings representing the image. The sequence of image embeddings is provided as at least part of an input sequence to that is processed by a language model neural network.

公开(公告)号：US12008324B2

公开(公告)日：2024-06-11

申请号：US17744921

申请日：2022-05-16

Applicant: DeepMind Technologies Limited

Inventor： Karl Moritz Hermann ,  Philip Blunsom ,  Felix George Hill

IPC: G06F40/30 ,  G06F17/16 ,  G06N3/08

CPC classification number: G06F40/30 ,  G06F17/16 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent interacting with an environment. In one aspect, a system includes a language encoder model that is configured to receive a text string in a particular natural language, and process the text string to generate a text embedding of the text string. The system includes an observation encoder neural network that is configured to receive an observation characterizing a state of the environment, and process the observation to generate an observation embedding of the observation. The system includes a subsystem that is configured to obtain a current text embedding of a current text string and a current observation embedding of a current observation. The subsystem is configured to select an action to be performed by the agent in response to the current observation.