公开(公告)号：US11900558B2

公开(公告)日：2024-02-13

申请号：US17520071

申请日：2021-11-05

Applicant: Adobe Inc.

Inventor： Marissa Ramirez de Chanlatte ,  Radomir Mech ,  Matheus Gadelha ,  Thibault Groueix

IPC: G06T19/20 ,  G06N3/04 ,  G06T7/50 ,  G06F18/214 ,  G06T17/00

CPC classification number: G06T19/20 ,  G06F18/214 ,  G06N3/04 ,  G06T7/50 ,  G06T17/00 ,  G06T2219/2004

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that tune a 3D-object-reconstruction-machine-learning model to reconstruct 3D models of objects from real images using real images as training data. For instance, the disclosed systems can determine a depth map for a real two-dimensional (2D) image and then reconstruct a 3D model of a digital object in the real 2D image based on the depth map. By using a depth map for a real 2D image, the disclosed systems can generate reconstructed 3D models that better conform to the shape of digital objects in real images than existing systems and use such reconstructed 3D models to generate more realistic looking visual effects (e.g., shadows, relighting).

公开(公告)号：US20240161320A1

公开(公告)日：2024-05-16

申请号：US18055594

申请日：2022-11-15

Applicant: Adobe Inc.

Inventor： Matheus Gadelha ,  Radomir Mech

IPC: G06T7/55

CPC classification number: G06T7/55 ,  H04N2013/0074

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating three-dimensional meshes representing two-dimensional images for editing the two-dimensional images. The disclosed system utilizes a first neural network to determine density values of pixels of a two-dimensional image based on estimated disparity. The disclosed system samples points in the two-dimensional image according to the density values and generates a tessellation based on the sampled points. The disclosed system utilizes a second neural network to estimate camera parameters and modify the three-dimensional mesh based on the estimated camera parameters of the pixels of the two-dimensional image. In one or more additional embodiments, the disclosed system generates a three-dimensional mesh to modify a two-dimensional image according to a displacement input. Specifically, the disclosed system maps the three-dimensional mesh to the two-dimensional image, modifies the three-dimensional mesh in response to a displacement input, and updates the two-dimensional image.

公开(公告)号：US20240144586A1

公开(公告)日：2024-05-02

申请号：US18304179

申请日：2023-04-20

Applicant: Adobe Inc.

Inventor： Yannick Hold-Geoffroy ,  Vojtech Krs ,  Radomir Mech ,  Nathan Carr ,  Matheus Gadelha

IPC: G06T15/60

CPC classification number: G06T15/60 ,  G06T2215/12

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify two-dimensional images via scene-based editing using three-dimensional representations of the two-dimensional images. For instance, in one or more embodiments, the disclosed systems utilize three-dimensional representations of two-dimensional images to generate and modify shadows in the two-dimensional images according to various shadow maps. Additionally, the disclosed systems utilize three-dimensional representations of two-dimensional images to modify humans in the two-dimensional images. The disclosed systems also utilize three-dimensional representations of two-dimensional images to provide scene scale estimation via scale fields of the two-dimensional images. In some embodiments, the disclosed systems utilizes three-dimensional representations of two-dimensional images to generate and visualize 3D planar surfaces for modifying objects in two-dimensional images. The disclosed systems further use three-dimensional representations of two-dimensional images to customize focal points for the two-dimensional images.

公开(公告)号：US20240161366A1

公开(公告)日：2024-05-16

申请号：US18055584

申请日：2022-11-15

Applicant: Adobe Inc.

Inventor： Radomir Mech ,  Nathan Carr ,  Matheus Gadelha

IPC: G06T11/60 ,  G06T7/70 ,  G06T17/20 ,  G06T19/20

CPC classification number: G06T11/60 ,  G06T7/70 ,  G06T17/20 ,  G06T19/20 ,  G06T2219/2004

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating three-dimensional meshes representing two-dimensional images for editing the two-dimensional images. The disclosed system utilizes a first neural network to determine density values of pixels of a two-dimensional image based on estimated disparity. The disclosed system samples points in the two-dimensional image according to the density values and generates a tessellation based on the sampled points. The disclosed system utilizes a second neural network to estimate camera parameters and modify the three-dimensional mesh based on the estimated camera parameters of the pixels of the two-dimensional image. In one or more additional embodiments, the disclosed system generates a three-dimensional mesh to modify a two-dimensional image according to a displacement input. Specifically, the disclosed system maps the three-dimensional mesh to the two-dimensional image, modifies the three-dimensional mesh in response to a displacement input, and updates the two-dimensional image.

公开(公告)号：US20230274040A1

公开(公告)日：2023-08-31

申请号：US17683188

申请日：2022-02-28

Applicant: Adobe Inc.

Inventor： Adrien Kaiser ,  Vojtech Krs ,  Thibault Groueix ,  Tamy Boubekeur ,  Pierre Gueth ,  Mathieu Gaillard ,  Matheus Gadelha

IPC: G06F30/12 ,  G06T19/20

CPC classification number: G06F30/12 ,  G06T19/20 ,  G06T2219/2021

Abstract: Certain aspects and features of this disclosure relate to modeling shapes using differentiable, signed distance functions. 3D modeling software can edit a 3D model represented using the differentiable, signed distance functions while displaying the model in a manner that is computing resource efficient and fast. Further, such 3D modeling software can automatically create such an editable 3D model from a reference representation that can be obtained in various ways and stored in a variety of formats. For example, a real-world object can be scanned using LiDAR and a reference representation can be produced from the LiDAR data. Candidate procedural models from a library of curated procedural models are optimized to obtain the best procedural model for editing. A selected procedural model provides an editable, reconstructed shape based on the reference representation of the object.

公开(公告)号：US20230147722A1

公开(公告)日：2023-05-11

申请号：US17520071

申请日：2021-11-05

Applicant: Adobe Inc.

Inventor： Marissa Ramirez de Chanlatte ,  Radomir Mech ,  Matheus Gadelha ,  Thibault Groueix

IPC: G06T19/20 ,  G06K9/62 ,  G06T7/50 ,  G06N3/04

CPC classification number: G06T19/20 ,  G06K9/6256 ,  G06T7/50 ,  G06N3/04 ,  G06T2219/2004

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that tune a 3D-object-reconstruction-machine-learning model to reconstruct 3D models of objects from real images using real images as training data. For instance, the disclosed systems can determine a depth map for a real two-dimensional (2D) image and then reconstruct a 3D model of a digital object in the real 2D image based on the depth map. By using a depth map for a real 2D image, the disclosed systems can generate reconstructed 3D models that better conform to the shape of digital objects in real images than existing systems and use such reconstructed 3D models to generate more realistic looking visual effects (e.g., shadows, relighting).

公开(公告)号：US12277652B2

公开(公告)日：2025-04-15

申请号：US18055585

申请日：2022-11-15

Applicant: Adobe Inc.

Inventor： Radomir Mech ,  Nathan Carr ,  Matheus Gadelha

IPC: G06T17/20 ,  G06T7/11 ,  G06T7/50 ,  G06T7/70 ,  G06T11/60 ,  G06V20/70

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating three-dimensional meshes representing two-dimensional images for editing the two-dimensional images. The disclosed system utilizes a first neural network to determine density values of pixels of a two-dimensional image based on estimated disparity. The disclosed system samples points in the two-dimensional image according to the density values and generates a tessellation based on the sampled points. The disclosed system utilizes a second neural network to estimate camera parameters and modify the three-dimensional mesh based on the estimated camera parameters of the pixels of the two-dimensional image. In one or more additional embodiments, the disclosed system generates a three-dimensional mesh to modify a two-dimensional image according to a displacement input. Specifically, the disclosed system maps the three-dimensional mesh to the two-dimensional image, modifies the three-dimensional mesh in response to a displacement input, and updates the two-dimensional image.

公开(公告)号：US20250061660A1

公开(公告)日：2025-02-20

申请号：US18451961

申请日：2023-08-18

Applicant: ADOBE INC.

Inventor： Ta-Ying Cheng ,  Matheus Gadelha ,  Soren Pirk ,  Radomir Mech ,  Thibault Groueix

IPC: G06T17/20 ,  G06V10/46 ,  G06V10/762

Abstract: Systems and methods for extracting 3D shapes from unstructured and unannotated datasets are described. Embodiments are configured to obtain a first image and a second image, where the first image depicts an object and the second image includes a corresponding object of a same object category as the object. Embodiments are further configured to generate, using an image encoder, image features for portions of the first image and for portions of the second image; identify a keypoint correspondence between a first keypoint in the first image and a second keypoint in the second image by clustering the image features corresponding to the portions of the first image and the portions of the second image; and generate, using an occupancy network, a 3D model of the object based on the keypoint correspondence.

公开(公告)号：US20250061650A1

公开(公告)日：2025-02-20

申请号：US18451267

申请日：2023-08-17

Applicant: ADOBE INC.

Inventor： Matheus Gadelha ,  Tomasz Opasinski ,  Kevin James Blackburn-Matzen ,  Mathieu Kevin Pascal Gaillard ,  Giorgio Gori ,  Radomir Mech

IPC: G06T17/00 ,  G06F3/0482 ,  G06T7/50 ,  G06T19/00 ,  G06T19/20

Abstract: An image processing system is configured to receive a three-dimensional (3D) model and a text prompt that describes a scene corresponding to the 3D model. The system may then generate a depth map of the 3D model and generate an output image based on the depth map and the text prompt. The output image may depicts a view of the scene that includes textures described by the text prompt. The output image may be generated using an image generation model.

公开(公告)号：US20240161406A1

公开(公告)日：2024-05-16

申请号：US18055590

申请日：2022-11-15

Applicant: Adobe Inc.

Inventor： Radomir Mech ,  Nathan Carr ,  Matheus Gadelha

IPC: G06T17/20 ,  G06T5/00

CPC classification number: G06T17/205 ,  G06T5/005 ,  G06T2207/20084

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating three-dimensional meshes representing two-dimensional images for editing the two-dimensional images. The disclosed system utilizes a first neural network to determine density values of pixels of a two-dimensional image based on estimated disparity. The disclosed system samples points in the two-dimensional image according to the density values and generates a tessellation based on the sampled points. The disclosed system utilizes a second neural network to estimate camera parameters and modify the three-dimensional mesh based on the estimated camera parameters of the pixels of the two-dimensional image. In one or more additional embodiments, the disclosed system generates a three-dimensional mesh to modify a two-dimensional image according to a displacement input. Specifically, the disclosed system maps the three-dimensional mesh to the two-dimensional image, modifies the three-dimensional mesh in response to a displacement input, and updates the two-dimensional image.