公开(公告)号：US20220292701A1

公开(公告)日：2022-09-15

申请号：US17826849

申请日：2022-05-27

Applicant: Google LLC

Inventor： Reza Mahjourian ,  Martin Wicke ,  Anelia Angelova

IPC: G06T7/579 ,  G06T7/285 ,  G06N3/04 ,  G06N3/08

Abstract: A system includes a neural network implemented by one or more computers, in which the neural network includes an image depth prediction neural network and a camera motion estimation neural network. The neural network is configured to receive a sequence of images. The neural network is configured to process each image in the sequence of images using the image depth prediction neural network to generate, for each image, a respective depth output that characterizes a depth of the image, and to process a subset of images in the sequence of images using the camera motion estimation neural network to generate a camera motion output that characterizes the motion of a camera between the images in the subset. The image depth prediction neural network and the camera motion estimation neural network have been jointly trained using an unsupervised learning technique.

公开(公告)号：US20220189154A1

公开(公告)日：2022-06-16

申请号：US17605783

申请日：2020-05-22

Applicant: Google LLC

Inventor： Michael Sahngwon Ryoo ,  Anthony Jacob Piergiovanni ,  Mingxing Tan ,  Anelia Angelova

IPC: G06V10/82 ,  G06N3/04 ,  G06T7/207 ,  G06T1/20 ,  G06T3/40 ,  G06V10/776

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining one or more neural network architectures of a neural network for performing a video processing neural network task. In one aspect, a method comprises: at each of a plurality of iterations: selecting a parent neural network architecture from a set of neural network architectures; training a neural network having the parent neural network architecture to perform the video processing neural network task, comprising determining trained values of connection weight parameters of the parent neural network architecture; generating a new neural network architecture based at least in part on the trained values of the connection weight parameters of the parent neural network architecture; and adding the new neural network architecture to the set of neural network architectures.

公开(公告)号：US11348268B2

公开(公告)日：2022-05-31

申请号：US17010967

申请日：2020-09-03

Applicant: Google LLC

Inventor： Reza Mahjourian ,  Martin Wicke ,  Anelia Angelova

IPC: G06K9/00 ,  G06T7/579 ,  G06T7/285 ,  G06N3/04 ,  G06N3/08

Abstract: A system includes a neural network implemented by one or more computers, in which the neural network includes an image depth prediction neural network and a camera motion estimation neural network. The neural network is configured to receive a sequence of images. The neural network is configured to process each image in the sequence of images using the image depth prediction neural network to generate, for each image, a respective depth output that characterizes a depth of the image, and to process a subset of images in the sequence of images using the camera motion estimation neural network to generate a camera motion output that characterizes the motion of a camera between the images in the subset. The image depth prediction neural network and the camera motion estimation neural network have been jointly trained using an unsupervised learning technique.

公开(公告)号：US20210279511A1

公开(公告)日：2021-09-09

申请号：US17194090

申请日：2021-03-05

Applicant: Google LLC

Inventor： Ariel Gordon ,  Soeren Pirk ,  Anelia Angelova ,  Vincent Michael Casser ,  Yao Lu ,  Anthony Brohan ,  Zhao Chen ,  Jan Dlabal

IPC: G06K9/62 ,  G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network using consistency measures. One of the methods includes processing a particular training example from a mediator training data set using a first neural network to generate a first output for a first machine learning task; processing the particular training example in the mediator training data set using each of one or more second neural networks, wherein each second neural network is configured to generate a second output for a respective second machine learning task; determining, for each second machine learning task, a consistency target output for the first machine learning task; determining, for each second machine learning task, an error between the first output and the consistency target output corresponding to the second machine learning task; and generating a parameter update for the first neural network from the determined errors.

公开(公告)号：US20210073997A1

公开(公告)日：2021-03-11

申请号：US16562819

申请日：2019-09-06

Applicant: Google LLC

Inventor： Suhani Vora ,  Reza Mahjourian ,  Soeren Pirk ,  Anelia Angelova

IPC: G06T7/11 ,  G06T7/246 ,  G06T7/55 ,  G06K9/00 ,  G06N3/08

Abstract: This disclosure describes a system including one or more computers and one or more non-transitory storage devices storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations for generating a predicted segmentation map for potential objects in a future scene depicted in a future image. The operations includes: receiving a sequence of input images that depict the same scene, the input images being captured by a camera at different time steps, the sequence of input images comprising a current input image and one or more input images preceding the current image in the sequence; processing the current input image to generate a segmentation map for potential objects in the current input image and a respective depth map for the current input image; generating a point cloud for the current input image using the segmentation map and the depth map of the current input image, wherein the point cloud is a 3-dimensional (3D) structure representation of the scene as depicted in the current input image; processing the sequence of input images using an ego-motion estimation neural network to generate, for each pair of two consecutive input images in the sequence, a respective ego-motion output that characterizes motion of the camera between the two consecutive input images; processing the ego-motion outputs using a future ego-motion prediction neural network to generate a future ego-motion output that is a prediction of future motion of the camera from the current input image in the sequence to a future image, wherein the future image is an image that would be captured by the camera at a future time step; processing the point cloud of the current input image and the future ego-motion output to generate a future point cloud that is a predicted 3D representation of a future scene as depicted in the future image; and processing the future point cloud to generate a predicted segmentation map for potential objects in the future scene depicted in the future image.

公开(公告)号：US20200258249A1

公开(公告)日：2020-08-13

申请号：US16861441

申请日：2020-04-29

Applicant: Google LLC

Inventor： Anelia Angelova ,  Martin Wicke ,  Reza Mahjourian

IPC: G06T7/579 ,  G06N3/08 ,  G06N3/04 ,  G06T7/285

Abstract: A system includes a neural network implemented by one or more computers, in which the neural network includes an image depth prediction neural network and a camera motion estimation neural network. The neural network is configured to receive a sequence of images. The neural network is configured to process each image in the sequence of images using the image depth prediction neural network to generate, for each image, a respective depth output that characterizes a depth of the image, and to process a subset of images in the sequence of images using the camera motion estimation neural network to generate a camera motion output that characterizes the motion of a camera between the images in the subset. The image depth prediction neural network and the camera motion estimation neural network have been jointly trained using an unsupervised learning technique.