公开(公告)号：US20240355109A1

公开(公告)日：2024-10-24

申请号：US18746977

申请日：2024-06-18

Applicant: Google LLC

Inventor： Michael Sahngwon Ryoo ,  Anthony Jacob Piergiovanni ,  Mingxing Tan ,  Anelia Angelova

IPC: G06V10/82 ,  G06N3/045 ,  G06T1/20 ,  G06T3/4046 ,  G06T7/207 ,  G06V10/776

CPC classification number: G06V10/82 ,  G06N3/045 ,  G06T1/20 ,  G06T3/4046 ,  G06T7/207 ,  G06V10/776 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining one or more neural network architectures of a neural network for performing a video processing neural network task. In one aspect, a method comprises: at each of a plurality of iterations: selecting a parent neural network architecture from a set of neural network architectures; training a neural network having the parent neural network architecture to perform the video processing neural network task, comprising determining trained values of connection weight parameters of the parent neural network architecture; generating a new neural network architecture based at least in part on the trained values of the connection weight parameters of the parent neural network architecture; and adding the new neural network architecture to the set of neural network architectures.

公开(公告)号：US12046025B2

公开(公告)日：2024-07-23

申请号：US17605783

申请日：2020-05-22

Applicant: Google LLC

Inventor： Michael Sahngwon Ryoo ,  Anthony Jacob Piergiovanni ,  Mingxing Tan ,  Anelia Angelova

IPC: G06V10/82 ,  G06N3/045 ,  G06T1/20 ,  G06T3/4046 ,  G06T7/207 ,  G06V10/776

CPC classification number: G06V10/82 ,  G06N3/045 ,  G06T1/20 ,  G06T3/4046 ,  G06T7/207 ,  G06V10/776 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining one or more neural network architectures of a neural network for performing a video processing neural network task. In one aspect, a method comprises: at each of a plurality of iterations: selecting a parent neural network architecture from a set of neural network architectures; training a neural network having the parent neural network architecture to perform the video processing neural network task, comprising determining trained values of connection weight parameters of the parent neural network architecture; generating a new neural network architecture based at least in part on the trained values of the connection weight parameters of the parent neural network architecture; and adding the new neural network architecture to the set of neural network architectures.

公开(公告)号：US20240189994A1

公开(公告)日：2024-06-13

申请号：US18539171

申请日：2023-12-13

Applicant: Google LLC

Inventor： Keerthana P G ,  Karol Hausman ,  Julian Ibarz ,  Brian Ichter ,  Alexander Irpan ,  Dmitry Kalashnikov ,  Yao Lu ,  Kanury Kanishka Rao ,  Michael Sahngwon Ryoo ,  Austin Charles Stone ,  Teddey Ming Xiao ,  Quan Ho Vuong ,  Sumedh Anand Sontakke

IPC: B25J9/16

CPC classification number: B25J9/163 ,  B25J9/161

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling an agent interacting with an environment. In one aspect, a method comprises: receiving a natural language text sequence that characterizes a task to be performed by the agent in the environment; generating an encoded representation of the natural language text sequence; and at each of a plurality of time steps: obtaining an observation image characterizing a state of the environment at the time step; processing the observation image to generate an encoded representation of the observation image; generating a sequence of input tokens; processing the sequence of input tokens to generate a policy output that defines an action to be performed by the agent in response to the observation image; selecting an action to be performed by the agent using the policy output; and causing the agent to perform the selected action.

公开(公告)号：US20230409899A1

公开(公告)日：2023-12-21

申请号：US17845753

申请日：2022-06-21

Applicant: Google LLC

Inventor： Michael Sahngwon Ryoo ,  Anthony Jacob Piergiovanni ,  Anelia Angelova ,  Anurag Arnab ,  Mostafa Dehghani

IPC: G06N3/08

CPC classification number: G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing a network input using a computer vision neural network with learned tokenization.

公开(公告)号：US20220366257A1

公开(公告)日：2022-11-17

申请号：US17620451

申请日：2020-09-16

Applicant: Google LLC

Inventor： Anthony J. Piergiovanni ,  Anelia Angelova ,  Michael Sahngwon Ryoo

IPC: G06N3/08 ,  G06K9/62 ,  G06N3/063 ,  G06N3/04 ,  H04N21/44

Abstract: Generally, the present disclosure is directed to a neural architecture search process for finding small and fast video processing networks for understanding of video data. The neural architecture search process can automatically design networks that provide comparable video processing performance at a fraction of the computational and storage cost of larger existing models, thereby conserving computing resources such as memory and processor usage.

公开(公告)号：US20230114556A1

公开(公告)日：2023-04-13

申请号：US17909581

申请日：2021-07-14

Applicant: Google LLC

Inventor： Michael Sahngwon Ryoo ,  Anthony Jacob Piergiovanni ,  Anelia Angelova

IPC: G06V10/82 ,  G06V10/26

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for processing a network input using a neural network to generate a network output. In one aspect, a method comprises processing a network input sing a neural network to generate a network output, where the neural network has multiple blocks, wherein each block is configured to process a block input to generate a block output, the method comprising, for each target block of the neural network: generating attention-weighted representations of multiple first block outputs, comprising, for each first block output: processing multiple second block outputs to generate attention factors; and generating the attention-weighted representation of each first block output by applying the respective attention factors to the corresponding first block output; and generating the target block input from the attention-weighted representations; and processing the target block input using the target block to generate a target block output.

公开(公告)号：US20220189154A1

公开(公告)日：2022-06-16

申请号：US17605783

申请日：2020-05-22

Applicant: Google LLC

Inventor： Michael Sahngwon Ryoo ,  Anthony Jacob Piergiovanni ,  Mingxing Tan ,  Anelia Angelova

IPC: G06V10/82 ,  G06N3/04 ,  G06T7/207 ,  G06T1/20 ,  G06T3/40 ,  G06V10/776

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining one or more neural network architectures of a neural network for performing a video processing neural network task. In one aspect, a method comprises: at each of a plurality of iterations: selecting a parent neural network architecture from a set of neural network architectures; training a neural network having the parent neural network architecture to perform the video processing neural network task, comprising determining trained values of connection weight parameters of the parent neural network architecture; generating a new neural network architecture based at least in part on the trained values of the connection weight parameters of the parent neural network architecture; and adding the new neural network architecture to the set of neural network architectures.