公开(公告)号：US20220327767A1

公开(公告)日：2022-10-13

申请号：US17807337

申请日：2022-06-16

Applicant: Adobe Inc.

Inventor： Tong He ,  John Collomosse ,  Hailin Jin

IPC: G06T15/08 ,  G06T7/73 ,  G06V10/40

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for utilizing an encoder-decoder architecture to learn a volumetric 3D representation of an object using digital images of the object from multiple viewpoints to render novel views of the object. For instance, the disclosed systems can utilize patch-based image feature extraction to extract lifted feature representations from images corresponding to different viewpoints of an object. Furthermore, the disclosed systems can model view-dependent transformed feature representations using learned transformation kernels. In addition, the disclosed systems can recurrently and concurrently aggregate the transformed feature representations to generate a 3D voxel representation of the object. Furthermore, the disclosed systems can sample frustum features using the 3D voxel representation and transformation kernels. Then, the disclosed systems can utilize a patch-based neural rendering approach to render images from frustum feature patches to display a view of the object from various viewpoints.

公开(公告)号：US11823322B2

公开(公告)日：2023-11-21

申请号：US17807337

申请日：2022-06-16

Applicant: Adobe Inc.

Inventor： Tong He ,  John Collomosse ,  Hailin Jin

IPC: G06T15/08 ,  G06T7/73 ,  G06V10/82 ,  G06V10/44 ,  G06V20/64

CPC classification number: G06T15/08 ,  G06T7/74 ,  G06V10/454 ,  G06V10/82 ,  G06V20/647 ,  G06T2200/08 ,  G06T2207/20084

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for utilizing an encoder-decoder architecture to learn a volumetric 3D representation of an object using digital images of the object from multiple viewpoints to render novel views of the object. For instance, the disclosed systems can utilize patch-based image feature extraction to extract lifted feature representations from images corresponding to different viewpoints of an object. Furthermore, the disclosed systems can model view-dependent transformed feature representations using learned transformation kernels. In addition, the disclosed systems can recurrently and concurrently aggregate the transformed feature representations to generate a 3D voxel representation of the object. Furthermore, the disclosed systems can sample frustum features using the 3D voxel representation and transformation kernels. Then, the disclosed systems can utilize a patch-based neural rendering approach to render images from frustum feature patches to display a view of the object from various viewpoints.

公开(公告)号：US20210312698A1

公开(公告)日：2021-10-07

申请号：US16838429

申请日：2020-04-02

Applicant: Adobe Inc.

Inventor： Tong He ,  John Collomosse ,  Hailin Jin

IPC: G06T15/08 ,  G06K9/46 ,  G06T7/73

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for utilizing an encoder-decoder architecture to learn a volumetric 3D representation of an object using digital images of the object from multiple viewpoints to render novel views of the object. For instance, the disclosed systems can utilize patch-based image feature extraction to extract lifted feature representations from images corresponding to different viewpoints of an object. Furthermore, the disclosed systems can model view-dependent transformed feature representations using learned transformation kernels. In addition, the disclosed systems can recurrently and concurrently aggregate the transformed feature representations to generate a 3D voxel representation of the object. Furthermore, the disclosed systems can sample frustum features using the 3D voxel representation and transformation kernels. Then, the disclosed systems can utilize a patch-based neural rendering approach to render images from frustum feature patches to display a view of the object from various viewpoints.