公开(公告)号：US20240338915A1

公开(公告)日：2024-10-10

申请号：US18132272

申请日：2023-04-07

申请人： Adobe Inc.

发明人： Zhixin Shu ,  Zexiang Xu ,  Shahrukh Athar ,  Sai Bi ,  Kalyan Sunkavalli ,  Fujun Luan

IPC分类号： G06T19/20 ,  G06N3/08 ,  G06T15/80 ,  G06T17/20

CPC分类号： G06T19/20 ,  G06N3/08 ,  G06T15/80 ,  G06T17/20 ,  G06T2210/44 ,  G06T2219/2012 ,  G06T2219/2021

摘要： Certain aspects and features of this disclosure relate to providing a controllable, dynamic appearance for neural 3D portraits. For example, a method involves projecting a color at points in a digital video portrait based on location, surface normal, and viewing direction for each respective point in a canonical space. The method also involves projecting, using the color, dynamic face normals for the points as changing according to an articulated head pose and facial expression in the digital video portrait. The method further involves disentangling, based on the dynamic face normals, a facial appearance in the digital video portrait into intrinsic components in the canonical space. The method additionally involves storing and/or rendering at least a portion of a head pose as a controllable, neural 3D portrait based on the digital video portrait using the intrinsic components.

公开(公告)号：US20240062495A1

公开(公告)日：2024-02-22

申请号：US17892097

申请日：2022-08-21

申请人： Adobe Inc.

发明人： Zhixin Shu ,  Zexiang Xu ,  Shahrukh Athar ,  Kalyan Sunkavalli ,  Elya Shechtman

IPC分类号： G06T19/20 ,  G06T17/00

CPC分类号： G06T19/20 ,  G06T17/00 ,  G06T2200/08 ,  G06T2219/2021

摘要： A scene modeling system receives a video including a plurality of frames corresponding to views of an object and a request to display an editable three-dimensional (3D) scene that corresponds to a particular frame of the plurality of frames. The scene modeling system applies a scene representation model to the particular frame, and includes a deformation model configured to generate, for each pixel of the particular frame based on a pose and an expression of the object, a deformation point using a 3D morphable model (3DMM) guided deformation field. The scene representation model includes a color model configured to determine, for the deformation point, color and volume density values. The scene modeling system receives a modification to one or more of the pose or the expression of the object including a modification to a location of the deformation point and renders an updated video based on the received modification.