Invention Publication
- Patent Title: UNSUPERVISED VOLUMETRIC ANIMATION
-
Application No.: US18089984Application Date: 2022-12-28
-
Publication No.: US20240221258A1Publication Date: 2024-07-04
- Inventor: Menglei Chai , Hsin-Ying Lee , Willi Menapace , Kyle Olszewski , Jian Ren , Aliaksandr Siarohin , Ivan Skorokhodov , Sergey Tulyakov
- Applicant: Menglei Chai , Hsin-Ying Lee , Willi Menapace , Kyle Olszewski , Jian Ren , Aliaksandr Siarohin , Ivan Skorokhodov , Sergey Tulyakov
- Applicant Address: US CA Los Angeles
- Assignee: Menglei Chai,Hsin-Ying Lee,Willi Menapace,Kyle Olszewski,Jian Ren,Aliaksandr Siarohin,Ivan Skorokhodov,Sergey Tulyakov
- Current Assignee: Menglei Chai,Hsin-Ying Lee,Willi Menapace,Kyle Olszewski,Jian Ren,Aliaksandr Siarohin,Ivan Skorokhodov,Sergey Tulyakov
- Current Assignee Address: US CA Los Angeles
- Main IPC: G06T13/40
- IPC: G06T13/40 ; G06T7/70 ; G06T19/20

Abstract:
Unsupervised volumetric 3D animation (UVA) of non-rigid deformable objects without annotations learns the 3D structure and dynamics of objects solely from single-view red/green/blue (RGB) videos and decomposes the single-view RGB videos into semantically meaningful parts that can be tracked and animated. Using a 3D autodecoder framework, paired with a keypoint estimator via a differentiable perspective-n-point (PnP) algorithm, the UVA model learns the underlying object 3D geometry and parts decomposition in an entirely unsupervised manner from still or video images. This allows the UVA model to perform 3D segmentation, 3D keypoint estimation, novel view synthesis, and animation. The UVA model can obtain animatable 3D objects from a single or a few images. The UVA method also features a space in which all objects are represented in their canonical, animation-ready form. Applications include the creation of lenses from images or videos for social media applications.
Information query