公开(公告)号：US20230351724A1

公开(公告)日：2023-11-02

申请号：US17800688

申请日：2020-02-18

Applicant: Google LLC

Inventor： Tingbo Hou ,  Adel Ahmadyan ,  Jianing Wei ,  Matthias Grundmann

IPC: G06V10/75 ,  G06V10/80 ,  G06V20/40 ,  G06T7/70

CPC classification number: G06V10/751 ,  G06V10/817 ,  G06V20/46 ,  G06T7/70 ,  G06V2201/12 ,  G06V2201/07 ,  G06T2207/20081

Abstract: The present disclosure is directed to systems and methods for performing object detection and pose estimation in 3D from 2D images. Object detection can be performed by a machine-learned model configured to determine various object properties. Implementations according to the disclosure can use these properties to estimate object pose and size.

公开(公告)号：US11770551B2

公开(公告)日：2023-09-26

申请号：US17122292

申请日：2020-12-15

Applicant: Google LLC

Inventor： Adel Ahmadyan ,  Tingbo Hou ,  Jianing Wei ,  Liangkai Zhang ,  Artsiom Ablavatski ,  Matthias Grundmann

IPC: G06V10/00 ,  H04N19/54 ,  H04N19/593 ,  H04N19/17 ,  H04N19/105 ,  H04N19/62 ,  G06V20/40

CPC classification number: H04N19/54 ,  G06V20/49 ,  H04N19/105 ,  H04N19/17 ,  H04N19/593 ,  H04N19/62

Abstract: A method includes receiving a video comprising images representing an object, and determining, using a machine learning model, based on a first image of the images, and for each respective vertex of vertices of a bounding volume for the object, first two-dimensional (2D) coordinates of the respective vertex. The method also includes tracking, from the first image to a second image of the images, a position of each respective vertex along a plane underlying the bounding volume, and determining, for each respective vertex, second 2D coordinates of the respective vertex based on the position of the respective vertex along the plane. The method further includes determining, for each respective vertex, (i) first three-dimensional (3D) coordinates of the respective vertex based on the first 2D coordinates and (ii) second 3D coordinates of the respective vertex based on the second 2D coordinates.

公开(公告)号：US20220415030A1

公开(公告)日：2022-12-29

申请号：US17778085

申请日：2019-11-19

Applicant: Tingbo HOU ,  Jianing WEI ,  Adel AHMADYAN ,  Matthias GRUNDMANN ,  Google LLC

Inventor： Tingbo Hou ,  Jianing Wei ,  Adel Ahmadyan ,  Matthias Grundmann

IPC: G06V10/774 ,  G06V20/64

Abstract: The present disclosure is directed to systems and methods for generating synthetic training data using augmented reality (AR) techniques. For example, images of a scene can be used to generate a three-dimensional mapping of the scene. The three-dimensional mapping may be associated with the images to indicate locations for positioning a virtual object. Using an AR rendering engine, implementations can generate an and orientation. The augmented image can then be stored in a machine learning dataset and associated with a label based on aspects of the virtual object.

公开(公告)号：US20220191542A1

公开(公告)日：2022-06-16

申请号：US17122292

申请日：2020-12-15

Applicant: Google LLC

Inventor： Adel Ahmadyan ,  Tingbo Hou ,  Jianing Wei ,  Liangkai Zhang ,  Artsiom Ablavatski ,  Matthias Grundmann

IPC: H04N19/54 ,  G06K9/00 ,  H04N19/62 ,  H04N19/17 ,  H04N19/105 ,  H04N19/593

Abstract: A method includes receiving a video comprising images representing an object, and determining, using a machine learning model, based on a first image of the images, and for each respective vertex of vertices of a bounding volume for the object, first two-dimensional (2D) coordinates of the respective vertex. The method also includes tracking, from the first image to a second image of the images, a position of each respective vertex along a plane underlying the bounding volume, and determining, for each respective vertex, second 2D coordinates of the respective vertex based on the position of the respective vertex along the plane. The method further includes determining, for each respective vertex, (i) first three-dimensional (3D) coordinates of the respective vertex based on the first 2D coordinates and (ii) second 3D coordinates of the respective vertex based on the second 2D coordinates.

公开(公告)号：US11436755B2

公开(公告)日：2022-09-06

申请号：US16988683

申请日：2020-08-09

Applicant: Google LLC

Inventor： Tingbo Hou ,  Matthias Grundmann ,  Liangkai Zhang ,  Jianing Wei ,  Adel Ahmadyan

IPC: G06T7/73 ,  G06T7/11

Abstract: Example embodiments allow for fast, efficient determination of bounding box vertices or other pose information for objects based on images of a scene that may contain the objects. An artificial neural network or other machine learning algorithm is used to generate, from an input image, a heat map and a number of pairs of displacement maps. The location of a peak within the heat map is then used to extract, from the displacement maps, the two-dimensional displacement, from the location of the peak within the image, of vertices of a bounding box that contains the object. This bounding box can then be used to determine the pose of the object within the scene. The artificial neural network can be configured to generate intermediate segmentation maps, coordinate maps, or other information about the shape of the object so as to improve the estimated bounding box.