公开(公告)号：US20200265294A1

公开(公告)日：2020-08-20

申请号：US16276559

申请日：2019-02-14

Applicant: Adobe Inc.

Inventor： Vladimir Kim ,  Omid Poursaeed ,  Jun Saito ,  Elya Shechtman

IPC: G06N3/04 ,  G06T13/00 ,  G06T17/20 ,  G06N3/08

Abstract: In implementations of object animation using generative neural networks, one or more computing devices of a system implement an animation system for reproducing animation of an object in a digital video. A mesh of the object is obtained from a first frame of the digital video and a second frame of the digital video having the object is selected. Features of the object from the second frame are mapped to vertices of the mesh, and the mesh is warped based on the mapping. The warped mesh is rendered as an image by a neural renderer and compared to the object from the second frame to train a neural network. The rendered image is then refined by a generator of a generative adversarial network which includes a discriminator. The discriminator trains the generator to reproduce the object from the second frame as the refined image.

公开(公告)号：US20210264659A1

公开(公告)日：2021-08-26

申请号：US16799664

申请日：2020-02-24

Applicant: Adobe Inc.

Inventor： Vladimir Kim ,  Omid Poursaeed ,  Noam Aigerman ,  Matthew Fisher

IPC: G06T15/20 ,  G06N3/04 ,  G06N3/08 ,  G06K9/62

Abstract: Certain embodiments involve techniques for generating a 3D representation based on a provided 2D image of an object. An image generation system receives the 2D image representation and generates a multi-dimensional vector of the input that represents the image. The image generation system samples a set of points and provides the set of points and the multi-dimensional vector to a neural network that was trained to predict a 3D surface representing the image such that the 3D surface is consistent with a 3D surface of the object calculated using an implicit function for representing the image. The neural network predicts, based on the multi-dimensional vector and the set of points, the 3D surface representing the object.

公开(公告)号：US10977549B2

公开(公告)日：2021-04-13

申请号：US16276559

申请日：2019-02-14

Applicant: Adobe Inc.

Inventor： Vladimir Kim ,  Omid Poursaeed ,  Jun Saito ,  Elya Shechtman

IPC: G06T13/00 ,  G06N3/04 ,  G06N3/08 ,  G06T17/20

Abstract: In implementations of object animation using generative neural networks, one or more computing devices of a system implement an animation system for reproducing animation of an object in a digital video. A mesh of the object is obtained from a first frame of the digital video and a second frame of the digital video having the object is selected. Features of the object from the second frame are mapped to vertices of the mesh, and the mesh is warped based on the mapping. The warped mesh is rendered as an image by a neural renderer and compared to the object from the second frame to train a neural network. The rendered image is then refined by a generator of a generative adversarial network which includes a discriminator. The discriminator trains the generator to reproduce the object from the second frame as the refined image.