公开(公告)号：US20220261573A1

公开(公告)日：2022-08-18

申请号：US17175441

申请日：2021-02-12

Applicant: Adobe Inc.

Inventor： Jimei YANG ,  Deepali ANEJA ,  Dingzeyu LI ,  Jun SAITO ,  Yang ZHOU

IPC: G06K9/00 ,  H04N21/845 ,  H04N21/8547 ,  G06T7/215 ,  H04N5/06

Abstract: Embodiments are disclosed for re-timing a video sequence to an audio sequence based on the detection of motion beats in the video sequence and audio beats in the audio sequence. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a first input, the first input including a video sequence, detecting motion beats in the video sequence, receiving a second input, the second input including an audio sequence, detecting audio beats in the audio sequence, modifying the video sequence by matching the detected motions beats in the video sequence to the detected audio beats in the audio sequence, and outputting the modified video sequence.

公开(公告)号：US20220366546A1

公开(公告)日：2022-11-17

申请号：US17812639

申请日：2022-07-14

Applicant: ADOBE INC.

Inventor： Zhe LIN ,  Yu ZENG ,  Jimei YANG ,  Jianming ZHANG ,  Elya SHECHTMAN

IPC: G06T5/00 ,  G06N3/04 ,  G06N3/08

Abstract: Methods and systems are provided for accurately filling holes, regions, and/or portions of images using iterative image inpainting. In particular, iterative inpainting utilize a confidence analysis of predicted pixels determined during the iterations of inpainting. For instance, a confidence analysis can provide information that can be used as feedback to progressively fill undefined pixels that comprise the holes, regions, and/or portions of an image where information for those respective pixels is not known. To allow for accurate image inpainting, one or more neural networks can be used. For instance, a coarse result neural network (e.g., a GAN comprised of a generator and a discriminator) and a fine result neural network (e.g., a GAN comprised of a generator and two discriminators). The image inpainting system can use such networks to predict an inpainting image result that fills the hole, region, and/or portion of the image using predicted pixels and generates a corresponding confidence map of the predicted pixels.

公开(公告)号：US20210342984A1

公开(公告)日：2021-11-04

申请号：US16864388

申请日：2020-05-01

Applicant: ADOBE INC.

Inventor： Zhe LIN ,  Yu ZENG ,  Jimei YANG ,  Jianming ZHANG ,  Elya SHECHTMAN

IPC: G06T5/00 ,  G06T3/40

Abstract: Methods and systems are provided for accurately filling holes, regions, and/or portions of high-resolution images using guided upsampling during image inpainting. For instance, an image inpainting system can apply guided upsampling to an inpainted image result to enable generation of a high-resolution inpainting result from a lower-resolution image that has undergone inpainting. To allow for guided upsampling during image inpainting, one or more neural networks can be used. For instance, a low-resolution result neural network (e.g., comprised of an encoder and a decoder) and a high-resolution input neural network (e.g., comprised of an encoder and a decoder). The image inpainting system can use such networks to generate a high-resolution inpainting image result that fills the hole, region, and/or portion of the image.

公开(公告)号：US20240161335A1

公开(公告)日：2024-05-16

申请号：US18055310

申请日：2022-11-14

Applicant: Adobe Inc.

Inventor： Yang ZHOU ,  Jimei YANG ,  Jun SAITO ,  Dingzeyu LI ,  Deepali ANEJA

IPC: G06T7/73 ,  G06F16/683 ,  G06F40/242 ,  G06T7/207

CPC classification number: G06T7/73 ,  G06F16/685 ,  G06F40/242 ,  G06T7/207

Abstract: Embodiments are disclosed for generating a gesture reenactment video sequence corresponding to a target audio sequence using a trained network based on a video motion graph generated from a reference speech video. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a first input including a reference speech video and generating a video motion graph representing the reference speech video, where each node is associated with a frame of the reference video sequence and reference audio features of the reference audio sequence. The disclosed systems and methods further comprise receiving a second input including a target audio sequence, generating target audio features, identifying a node path through the video motion graph based on the target audio features and the reference audio features, and generating an output media sequence based on the identified node path through the video motion graph paired with the target audio sequence.

公开(公告)号：US20210342983A1

公开(公告)日：2021-11-04

申请号：US16861548

申请日：2020-04-29

Applicant: ADOBE INC.

Inventor： Zhe LIN ,  Yu ZENG ,  Jimei YANG ,  Jianming ZHANG ,  Elya SHECHTMAN

IPC: G06T5/00 ,  G06N3/04 ,  G06N3/08

Abstract: Methods and systems are provided for accurately filling holes, regions, and/or portions of images using iterative image inpainting. In particular, iterative inpainting utilize a confidence analysis of predicted pixels determined during the iterations of inpainting. For instance, a confidence analysis can provide information that can be used as feedback to progressively fill undefined pixels that comprise the holes, regions, and/or portions of an image where information for those respective pixels is not known. To allow for accurate image inpainting, one or more neural networks can be used. For instance, a coarse result neural network (e.g., a GAN comprised of a generator and a discriminator) and a fine result neural network (e.g., a GAN comprised of a generator and two discriminators). The image inpainting system can use such networks to predict an inpainting image result that fills the hole, region, and/or portion of the image using predicted pixels and generates a corresponding confidence map of the predicted pixels.