公开(公告)号：US20190130229A1

公开(公告)日：2019-05-02

申请号：US15799395

申请日：2017-10-31

Applicant: Adobe Inc.

Inventor： Xin Lu ,  Zhe Lin ,  Xiaohui Shen ,  Jimei Yang ,  Jianming Zhang ,  Jen-Chan Jeff Chien ,  Chenxi Liu

IPC: G06K9/66 ,  G06K9/46 ,  G06T7/194

CPC classification number: G06K9/66 ,  G06K9/4628 ,  G06K9/4671 ,  G06N3/0454 ,  G06N3/08 ,  G06T7/194 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for segmenting objects in digital visual media utilizing one or more salient content neural networks. In particular, in one or more embodiments, the disclosed systems and methods train one or more salient content neural networks to efficiently identify foreground pixels in digital visual media. Moreover, in one or more embodiments, the disclosed systems and methods provide a trained salient content neural network to a mobile device, allowing the mobile device to directly select salient objects in digital visual media utilizing a trained neural network. Furthermore, in one or more embodiments, the disclosed systems and methods train and provide multiple salient content neural networks, such that mobile devices can identify objects in real-time digital visual media feeds (utilizing a first salient content neural network) and identify objects in static digital images (utilizing a second salient content neural network).

公开(公告)号：US20240378809A1

公开(公告)日：2024-11-14

申请号：US18316490

申请日：2023-05-12

Applicant: Adobe Inc.

Inventor： Yangtuanfeng Wang ,  Yi Zhou ,  Yasamin Jafarian ,  Nathan Aaron Carr ,  Jimei Yang ,  Duygu Ceylan Aksit

IPC: G06T17/20

Abstract: Decal application techniques as implemented by a computing device are described to perform decaling of a digital image. In one example, learned features of a digital image using machine learning are used by a computing device as a basis to predict the surface geometry of an object in the digital image. Once the surface geometry of the object is predicted, machine learning techniques are then used by the computing device to configure an overlay object to be applied onto the digital image according to the predicted surface geometry of the overlaid object.

公开(公告)号：US12067680B2

公开(公告)日：2024-08-20

申请号：US17816813

申请日：2022-08-02

Applicant: ADOBE INC.

Inventor： Jimei Yang ,  Chun-han Yao ,  Duygu Ceylan Aksit ,  Yi Zhou

IPC: G06T17/20

CPC classification number: G06T17/20

Abstract: Systems and methods for mesh generation are described. One aspect of the systems and methods includes receiving an image depicting a visible portion of a body; generating an intermediate mesh representing the body based on the image; generating visibility features indicating whether parts of the body are visible based on the image; generating parameters for a morphable model of the body based on the intermediate mesh and the visibility features; and generating an output mesh representing the body based on the parameters for the morphable model, wherein the output mesh includes a non-visible portion of the body that is not depicted by the image.

公开(公告)号：US20240169553A1

公开(公告)日：2024-05-23

申请号：US18057436

申请日：2022-11-21

Applicant: Adobe Inc.

Inventor： Jae shin Yoon ,  Zhixin Shu ,  Yangtuanfeng Wang ,  Jingwan Lu ,  Jimei Yang ,  Duygu Ceylan Aksit

IPC: G06T7/20 ,  G06T13/40 ,  G06T15/04 ,  G06T17/00

CPC classification number: G06T7/20 ,  G06T13/40 ,  G06T15/04 ,  G06T17/00 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30244

Abstract: Techniques for modeling secondary motion based on three-dimensional models are described as implemented by a secondary motion modeling system, which is configured to receive a plurality of three-dimensional object models representing an object. Based on the three-dimensional object models, the secondary motion modeling system determines three-dimensional motion descriptors of a particular three-dimensional object model using one or more machine learning models. Based on the three-dimensional motion descriptors, the secondary motion modeling system models at least one feature subjected to secondary motion using the one or more machine learning models. The particular three-dimensional object model having the at least one feature is rendered by the secondary motion modeling system.

公开(公告)号：US20240135513A1

公开(公告)日：2024-04-25

申请号：US18190654

申请日：2023-03-27

Applicant: Adobe Inc.

Inventor： Krishna Kumar Singh ,  Yijun Li ,  Jingwan Lu ,  Duygu Ceylan Aksit ,  Yangtuanfeng Wang ,  Jimei Yang ,  Tobias Hinz

IPC: G06T5/00 ,  G06T3/00 ,  G06T7/40 ,  G06T7/70 ,  G06V10/44 ,  G06V10/771 ,  G06V10/80 ,  G06V10/82

CPC classification number: G06T5/005 ,  G06T3/0093 ,  G06T7/40 ,  G06T7/70 ,  G06V10/44 ,  G06V10/771 ,  G06V10/806 ,  G06V10/82 ,  G06T2207/30196

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.

公开(公告)号：US11721056B2

公开(公告)日：2023-08-08

申请号：US17573890

申请日：2022-01-12

Applicant: Adobe Inc.

Inventor： Jimei Yang ,  Davis Rempe ,  Bryan Russell ,  Aaron Hertzmann

IPC: G06T13/40 ,  G06T15/20 ,  G06T15/50 ,  G06N3/08

CPC classification number: G06T13/40 ,  G06N3/08 ,  G06T15/20 ,  G06T15/50

Abstract: In some embodiments, a model training system obtains a set of animation models. For each of the animation models, the model training system renders the animation model to generate a sequence of video frames containing a character using a set of rendering parameters and extracts joint points of the character from each frame of the sequence of video frames. The model training system further determines, for each frame of the sequence of video frames, whether a subset of the joint points are in contact with a ground plane in a three-dimensional space and generates contact labels for the subset of the joint points. The model training system trains a contact estimation model using training data containing the joint points extracted from the sequences of video frames and the generated contact labels. The contact estimation model can be used to refine a motion model for a character.

公开(公告)号：US11704865B2

公开(公告)日：2023-07-18

申请号：US17383294

申请日：2021-07-22

Applicant: Adobe Inc.

Inventor： Ruben Villegas ,  Yunseok Jang ,  Duygu Ceylan Aksit ,  Jimei Yang ,  Xin Sun

IPC: G06T15/50 ,  G06T15/20 ,  G06T15/04 ,  G06T15/06 ,  G06T7/70 ,  G06N3/04

CPC classification number: G06T15/506 ,  G06N3/04 ,  G06T7/70 ,  G06T15/04 ,  G06T15/06 ,  G06T15/20

Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that generate realistic shading for three-dimensional objects inserted into digital images. The disclosed system utilizes a light encoder neural network to generate a representation embedding of lighting in a digital image. Additionally, the disclosed system determines points of the three-dimensional object visible within a camera view. The disclosed system generates a self-occlusion map for the digital three-dimensional object by determining whether fixed sets of rays uniformly sampled from the points intersects with the digital three-dimensional object. The disclosed system utilizes a generator neural network to determine a shading map for the digital three-dimensional object based on the representation embedding of lighting in the digital image and the self-occlusion map. Additionally, the disclosed system generates a modified digital image with the three-dimensional object inserted into the digital image with consistent lighting of the three-dimensional object and the digital image.

公开(公告)号：US11605156B2

公开(公告)日：2023-03-14

申请号：US17812639

申请日：2022-07-14

Applicant: ADOBE INC.

Inventor： Zhe Lin ,  Yu Zeng ,  Jimei Yang ,  Jianming Zhang ,  Elya Shechtman

IPC: G06T5/00 ,  G06N3/04 ,  G06N3/08

Abstract: Methods and systems are provided for accurately filling holes, regions, and/or portions of images using iterative image inpainting. In particular, iterative inpainting utilize a confidence analysis of predicted pixels determined during the iterations of inpainting. For instance, a confidence analysis can provide information that can be used as feedback to progressively fill undefined pixels that comprise the holes, regions, and/or portions of an image where information for those respective pixels is not known. To allow for accurate image inpainting, one or more neural networks can be used. For instance, a coarse result neural network (e.g., a GAN comprised of a generator and a discriminator) and a fine result neural network (e.g., a GAN comprised of a generator and two discriminators). The image inpainting system can use such networks to predict an inpainting image result that fills the hole, region, and/or portion of the image using predicted pixels and generates a corresponding confidence map of the predicted pixels.

公开(公告)号：US20210335028A1

公开(公告)日：2021-10-28

申请号：US16860411

申请日：2020-04-28

Applicant: Adobe Inc.

Inventor： Jimei Yang ,  Davis Rempe ,  Bryan Russell ,  Aaron Hertzmann

IPC: G06T13/40 ,  G06N3/08 ,  G06T15/50 ,  G06T15/20

Abstract: In some embodiments, a motion model refinement system receives an input video depicting a human character and an initial motion model describing motions of individual joint points of the human character in a three-dimensional space. The motion model refinement system identifies foot joint points of the human character that are in contact with a ground plane using a trained contact estimation model. The motion model refinement system determines the ground plane based on the foot joint points and the initial motion model and constructs an optimization problem for refining the initial motion model. The optimization problem minimizes the difference between the refined motion model and the initial motion model under a set of plausibility constraints including constraints on the contact foot joint points and a time-dependent inertia tensor-based constraint. The motion model refinement system obtains the refined motion model by solving the optimization problem.

公开(公告)号：US11017586B2

公开(公告)日：2021-05-25

申请号：US16388187

申请日：2019-04-18

Applicant: ADOBE INC.

Inventor： Mai Long ,  Simon Niklaus ,  Jimei Yang

IPC: G06T15/20 ,  G06T7/10 ,  G06N3/04

Abstract: Systems and methods are described for generating a three dimensional (3D) effect from a two dimensional (2D) image. The methods may include generating a depth map based on a 2D image, identifying a camera path, generating one or more extremal views based on the 2D image and the camera path, generating a global point cloud by inpainting occlusion gaps in the one or more extremal views, generating one or more intermediate views based on the global point cloud and the camera path, and combining the one or more extremal views and the one or more intermediate views to produce a 3D motion effect.