公开(公告)号：US20210334993A1

公开(公告)日：2021-10-28

申请号：US16861034

申请日：2020-04-28

Applicant: Snap Inc.

Inventor： Oliver Woodford

IPC: G06T7/55 ,  G06T5/00 ,  G06T5/50

Abstract: Aspects of the present disclosure involve a system and a method for performing operations comprising: accessing a source image depicting a target structure; accessing one or more target images depicting at least a portion of the target structure; computing correspondence between a first set of pixels in the source image of a first portion of the target structure and a second set of pixels in the one or more target images of the first portion of the target structure, the correspondence being computed as a function of camera parameters that vary between the source image and the one or more target images; and generating a three-dimensional (3D) model of the target structure based on the correspondence between the first set of pixels in the source image and the second set of pixels in the one or more target images based on a joint optimization of target structure and camera parameters.

公开(公告)号：US10776993B1

公开(公告)日：2020-09-15

申请号：US16287993

申请日：2019-02-27

Applicant: Snap Inc.

Inventor： Yiming Chen ,  Oliver Woodford

IPC: G06T15/20 ,  H04N13/117 ,  G06F3/01 ,  G06T19/00

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for determining orientation or location of a device. The program and method include receiving data from one or more sensors of a device; generating a first spline that represents location of the device in a world coordinate frame based on the data received from the one or more sensors; generating a second spline that represents orientation of the device in the world coordinate frame based on the data received from the one or more sensors; computing a motion model that represents motion of the device over a continuous time interval based on the first and second splines; and determining an expected location or orientation of the device based on the motion model.

公开(公告)号：US12056792B2

公开(公告)日：2024-08-06

申请号：US17557834

申请日：2021-12-21

Applicant: Snap Inc.

Inventor： Jian Ren ,  Menglei Chai ,  Oliver Woodford ,  Kyle Olszewski ,  Sergey Tulyakov

IPC: G06T11/00 ,  G06N3/045 ,  G06T3/60 ,  G06T7/194 ,  G06V40/10

CPC classification number: G06T11/00 ,  G06N3/045 ,  G06T3/60 ,  G06T7/194 ,  G06V40/10 ,  G06T2207/20084 ,  G06T2207/30196

Abstract: Systems and methods herein describe a motion retargeting system. The motion retargeting system accesses a plurality of two-dimensional images comprising a person performing a plurality of body poses, extracts a plurality of implicit volumetric representations from the plurality of body poses, generates a three-dimensional warping field, the three-dimensional warping field configured to warp the plurality of implicit volumetric representations from a canonical pose to a target pose, and based on the three-dimensional warping field, generates a two-dimensional image of an artificial person performing the target pose.

公开(公告)号：US20220375111A1

公开(公告)日：2022-11-24

申请号：US17813887

申请日：2022-07-20

Applicant: Snap Inc.

Inventor： Oliver Woodford

IPC: G06T7/55 ,  G06T5/00 ,  G06T5/50

Abstract: Aspects of the present disclosure involve a system and a method for performing operations comprising: accessing a source image depicting a target structure; accessing one or more target images depicting at least a portion of the target structure; computing correspondence between a first set of pixels in the source image of a first portion of the target structure and a second set of pixels in the one or more target images of the first portion of the target structure, the correspondence being computed as a function of camera parameters that vary between the source image and the one or more target images; and generating a three-dimensional (3D) model of the target structure based on the correspondence between the first set of pixels in the source image and the second set of pixels in the one or more target images based on a joint optimization of target structure and camera parameters.

公开(公告)号：US11430142B2

公开(公告)日：2022-08-30

申请号：US16861034

申请日：2020-04-28

Applicant: Snap Inc.

Inventor： Oliver Woodford

IPC: G06T7/55 ,  G06T5/00 ,  G06T5/50

Abstract: Aspects of the present disclosure involve a system and a method for performing operations comprising: accessing a source image depicting a target structure; accessing one or more target images depicting at least a portion of the target structure; computing correspondence between a first set of pixels in the source image of a first portion of the target structure and a second set of pixels in the one or more target images of the first portion of the target structure, the correspondence being computed as a function of camera parameters that vary between the source image and the one or more target images; and generating a three-dimensional (3D) model of the target structure based on the correspondence between the first set of pixels in the source image and the second set of pixels in the one or more target images based on a joint optimization of target structure and camera parameters.

公开(公告)号：US20220207786A1

公开(公告)日：2022-06-30

申请号：US17557834

申请日：2021-12-21

Applicant: Snap Inc.

Inventor： Jian Ren ,  Menglei Chai ,  Oliver Woodford ,  Kyle Olszewski ,  Sergey Tulyakov

IPC: G06T11/00 ,  G06V40/10 ,  G06T7/194 ,  G06T3/60 ,  G06N3/04

Abstract: Systems and methods herein describe a motion retargeting system. The motion retargeting system accesses a plurality of two-dimensional images comprising a person performing a plurality of body poses, extracts a plurality of implicit volumetric representations from the plurality of body poses, generates a three-dimensional warping field, the three-dimensional warping field configured to warp the plurality of implicit volumetric representations from a canonical pose to a target pose, and based on the three-dimensional warping field, generates a two-dimensional image of an artificial person performing the target pose.

公开(公告)号：US11836835B2

公开(公告)日：2023-12-05

申请号：US17364218

申请日：2021-06-30

Applicant: Snap Inc.

Inventor： Menglei Chai ,  Jian Ren ,  Aliaksandr Siarohin ,  Sergey Tulyakov ,  Oliver Woodford

IPC: G06T13/00 ,  G06N3/08 ,  G06T7/20 ,  G06T11/00 ,  G06N3/045

CPC classification number: G06T13/00 ,  G06N3/045 ,  G06N3/08 ,  G06T7/20 ,  G06T11/001 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Systems and methods herein describe novel motion representations for animating articulated objects consisting of distinct parts. The described systems and method access source image data, identify driving image data to modify image feature data in the source image sequence data, generate, using an image transformation neural network, modified source image data comprising a plurality of modified source images depicting modified versions of the image feature data, the image transformation neural network being trained to identify, for each image in the source image data, a driving image from the driving image data, the identified driving image being implemented by the image transformation neural network to modify a corresponding source image in the source image data using motion estimation differences between the identified driving image and the corresponding source image, and stores the modified source image data.

公开(公告)号：US11682176B2

公开(公告)日：2023-06-20

申请号：US17456487

申请日：2021-11-24

Applicant: Snap Inc.

Inventor： Oliver Woodford

IPC: G06T19/00 ,  G06T7/73

CPC classification number: G06T19/006 ,  G06T7/73 ,  G06T2200/04 ,  G06T2200/08 ,  G06T2207/20084

Abstract: A correlation system can be implemented on a user device (e.g., smartphone) to perform image processing tasks e.g., point to point correlations for tasks such as alignment, tracking, and reconstruction of model data. The correlation system can implement normalized cross correlation in a least squares optimization scheme without use of approximations. Normalized cross correlation optimized via least squares can further implement global and local cost functions that are invariant to dynamic lighting conditions.

公开(公告)号：US20220207329A1

公开(公告)日：2022-06-30

申请号：US17558327

申请日：2021-12-21

Applicant: Snap Inc.

Inventor： Jian Ren ,  Oliver Woodford ,  Sergey Tulyakov ,  Jiazhuo Wang ,  Qing Jin

IPC: G06N3/04 ,  G06N3/08

Abstract: Systems and methods herein describe an image compression system. The image compression system generates a first generative adversarial network (GAN), identifies a threshold, based on the threshold, generates a second GAN by pruning channels of the first GAN, trains the second GAN using similarity-based knowledge distillation from the first GAN, and stores the trained second GAN.

公开(公告)号：US11217022B1

公开(公告)日：2022-01-04

申请号：US16457607

申请日：2019-06-28

Applicant: Snap Inc.

Inventor： Oliver Woodford

IPC: G06T19/00 ,  G06T7/73

Abstract: A correlation system can be implemented on a user device (e.g., smartphone) to perform image processing tasks e.g., point to point correlations for tasks such as alignment, tracking, and reconstruction of model data. The correlation system can implement normalized cross correlation in a least squares optimization scheme without use of approximations. Normalized cross correlation optimized via least squares can further implement global and local cost functions that are invariant to dynamic lighting conditions.