公开(公告)号：US20240135512A1

公开(公告)日：2024-04-25

申请号：US18190556

申请日：2023-03-27

Applicant: Adobe Inc.

Inventor： Krishna Kumar Singh ,  Yijun Li ,  Jingwan Lu ,  Duygu Ceylan Aksit ,  Yangtuanfeng Wang ,  Jimei Yang ,  Tobias Hinz ,  Qing Liu ,  Jianming Zhang ,  Zhe Lin

IPC: G06T5/00 ,  G06T7/11 ,  G06V10/82 ,  G06V40/10

CPC classification number: G06T5/005 ,  G06T7/11 ,  G06V10/82 ,  G06V40/10 ,  G06T2207/20021 ,  G06T2207/20084 ,  G06T2207/20212 ,  G06T2207/30196

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.

公开(公告)号：US20240135511A1

公开(公告)日：2024-04-25

申请号：US18190544

申请日：2023-03-27

Applicant: Adobe Inc.

Inventor： Krishna Kumar Singh ,  Yijun Li ,  Jingwan Lu ,  Duygu Ceylan Aksit ,  Yangtuanfeng Wang ,  Jimei Yang ,  Tobias Hinz ,  Qing Liu ,  Jianming Zhang ,  Zhe Lin

IPC: G06T5/00 ,  G06V10/25 ,  G06V10/44 ,  G06V10/82

CPC classification number: G06T5/005 ,  G06V10/25 ,  G06V10/44 ,  G06V10/82 ,  G06T2207/30196

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.

公开(公告)号：US11948281B2

公开(公告)日：2024-04-02

申请号：US16864388

申请日：2020-05-01

Applicant: ADOBE INC.

Inventor： Zhe Lin ,  Yu Zeng ,  Jimei Yang ,  Jianming Zhang ,  Elya Shechtman

IPC: G06T5/00 ,  G06T3/40 ,  G06T3/4046 ,  G06T3/4076

CPC classification number: G06T5/005 ,  G06T3/4046 ,  G06T3/4076

Abstract: Methods and systems are provided for accurately filling holes, regions, and/or portions of high-resolution images using guided upsampling during image inpainting. For instance, an image inpainting system can apply guided upsampling to an inpainted image result to enable generation of a high-resolution inpainting result from a lower-resolution image that has undergone inpainting. To allow for guided upsampling during image inpainting, one or more neural networks can be used. For instance, a low-resolution result neural network (e.g., comprised of an encoder and a decoder) and a high-resolution input neural network (e.g., comprised of an encoder and a decoder). The image inpainting system can use such networks to generate a high-resolution inpainting image result that fills the hole, region, and/or portion of the image.

公开(公告)号：US11861779B2

公开(公告)日：2024-01-02

申请号：US17550432

申请日：2021-12-14

Applicant: Adobe Inc.

Inventor： Jun Saito ,  Jimei Yang ,  Duygu Ceylan Aksit

IPC: G06F3/01 ,  G06T13/80 ,  G06T7/33

CPC classification number: G06T13/80 ,  G06F3/012 ,  G06F3/017 ,  G06T7/33

Abstract: Digital object animation techniques are described. In a first example, translation-based animation of the digital object operates using control points of the digital object. In another example, the animation system is configured to minimize an amount of feature positions that are used to generate the animation. In a further example, an input pose is normalized through use of a global scale factor to address changes in a z-position of a subject in different digital images. Yet further, a body tracking module is used to computing initial feature positions. The initial feature positions are then used to initialize a face tracker module to generate feature positions of the face. The animation system also supports a plurality of modes used to generate the digital object, techniques to define a base of the digital object, and a friction term limiting movement of features positions based on contact with a ground plane.

公开(公告)号：US11682238B2

公开(公告)日：2023-06-20

申请号：US17175441

申请日：2021-02-12

Applicant: Adobe Inc.

Inventor： Jimei Yang ,  Deepali Aneja ,  Dingzeyu Li ,  Jun Saito ,  Yang Zhou

IPC: G06V40/20 ,  G06T7/215 ,  G06V20/40 ,  G06V40/10 ,  H04N5/06 ,  H04N21/8547 ,  G11B27/031 ,  G10H1/36 ,  G11B27/10 ,  H04N21/845

CPC classification number: G06V40/23 ,  G06T7/215 ,  G06V20/41 ,  G06V20/46 ,  G06V40/103 ,  H04N5/06 ,  H04N21/8456 ,  H04N21/8547

Abstract: Embodiments are disclosed for re-timing a video sequence to an audio sequence based on the detection of motion beats in the video sequence and audio beats in the audio sequence. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a first input, the first input including a video sequence, detecting motion beats in the video sequence, receiving a second input, the second input including an audio sequence, detecting audio beats in the audio sequence, modifying the video sequence by matching the detected motions beats in the video sequence to the detected audio beats in the audio sequence, and outputting the modified video sequence.

公开(公告)号：US11531697B2

公开(公告)日：2022-12-20

申请号：US17087982

申请日：2020-11-03

Applicant: Adobe Inc.

Inventor： Jinrong Xie ,  Shabnam Ghadar ,  Jun Saito ,  Jimei Yang ,  Elnaz Morad ,  Duygu Ceylan Aksit ,  Baldo Faieta ,  Alex Filipkowski

IPC: G06F16/55 ,  G06F16/538 ,  G06F16/583 ,  G06F16/56 ,  G06F16/535 ,  G06N3/08 ,  G06T7/73

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately, efficiently, and flexibly identifying and providing digital images of human figures in poses corresponding to a query pose. In particular, the disclosed systems can provide multiple approaches to searching for and providing pose images, including identifying a digital image depicting a human figure in a particular pose based on a query digital image that depicts the pose or identifying a digital image depicting a human figure in a particular pose based on a virtual mannequin. Indeed, the disclosed systems can provide a manipulable virtual mannequin that defines a query pose for searching a repository of digital images. Additionally, the disclosed systems can generate and provide digital pose image groups by clustering digital images together according to poses of human figures within a pose feature space.

公开(公告)号：US11436775B2

公开(公告)日：2022-09-06

申请号：US16806344

申请日：2020-03-02

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xin Lu ,  Xiaohui Shen ,  Jimei Yang ,  Jiahui Yu

IPC: G06T11/60 ,  G06K9/62 ,  G06T5/00 ,  G06T5/30 ,  G06N3/04 ,  G06N3/08 ,  G06V10/44 ,  G06V10/75 ,  G06V30/194

Abstract: Predicting patch displacement maps using a neural network is described. Initially, a digital image on which an image editing operation is to be performed is provided as input to a patch matcher having an offset prediction neural network. From this image and based on the image editing operation for which this network is trained, the offset prediction neural network generates an offset prediction formed as a displacement map, which has offset vectors that represent a displacement of pixels of the digital image to different locations for performing the image editing operation. Pixel values of the digital image are copied to the image pixels affected by the operation.

公开(公告)号：US20220139019A1

公开(公告)日：2022-05-05

申请号：US17573890

申请日：2022-01-12

Applicant: Adobe Inc.

Inventor： Jimei Yang ,  Davis Rempe ,  Bryan Russell ,  Aaron Hertzmann

IPC: G06T13/40 ,  G06N3/08 ,  G06T15/50 ,  G06T15/20

Abstract: In some embodiments, a model training system obtains a set of animation models. For each of the animation models, the model training system renders the animation model to generate a sequence of video frames containing a character using a set of rendering parameters and extracts joint points of the character from each frame of the sequence of video frames. The model training system further determines, for each frame of the sequence of video frames, whether a subset of the joint points are in contact with a ground plane in a three-dimensional space and generates contact labels for the subset of the joint points. The model training system trains a contact estimation model using training data containing the joint points extracted from the sequences of video frames and the generated contact labels. The contact estimation model can be used to refine a motion model for a character.

公开(公告)号：US20220138249A1

公开(公告)日：2022-05-05

申请号：US17087982

申请日：2020-11-03

Applicant: Adobe Inc.

Inventor： Jinrong Xie ,  Shabnam Ghadar ,  Jun Saito ,  Jimei Yang ,  Elnaz Morad ,  Duygu Ceylan Aksit ,  Baldo Faieta ,  Alex Filipkowski

IPC: G06F16/55 ,  G06F16/538 ,  G06N3/08 ,  G06F16/535 ,  G06T7/73 ,  G06F16/56 ,  G06F16/583

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately, efficiently, and flexibly identifying and providing digital images of human figures in poses corresponding to a query pose. In particular, the disclosed systems can provide multiple approaches to searching for and providing pose images, including identifying a digital image depicting a human figure in a particular pose based on a query digital image that depicts the pose or identifying a digital image depicting a human figure in a particular pose based on a virtual mannequin. Indeed, the disclosed systems can provide a manipulable virtual mannequin that defines a query pose for searching a repository of digital images. Additionally, the disclosed systems can generate and provide digital pose image groups by clustering digital images together according to poses of human figures within a pose feature space.

公开(公告)号：US11321847B2

公开(公告)日：2022-05-03

申请号：US17103119

申请日：2020-11-24

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Wei Xiong ,  Connelly Barnes ,  Jimei Yang ,  Xin Lu

IPC: G06T7/149 ,  G06T7/13 ,  G06T7/194 ,  G06T5/00

Abstract: In some embodiments, an image manipulation application receives an incomplete image that includes a hole area lacking image content. The image manipulation application applies a contour detection operation to the incomplete image to detect an incomplete contour of a foreground object in the incomplete image. The hole area prevents the contour detection operation from detecting a completed contour of the foreground object. The image manipulation application further applies a contour completion model to the incomplete contour and the incomplete image to generate the completed contour for the foreground object. Based on the completed contour and the incomplete image, the image manipulation application generates image content for the hole area to generate a completed image.