公开(公告)号：US20240046430A1

公开(公告)日：2024-02-08

申请号：US18375187

申请日：2023-09-29

Applicant: Adobe Inc.

Inventor： Oliver Wang ,  John Nelson ,  Geoffrey Oxholm ,  Elya Shechtman

IPC: G06T5/00 ,  G06T7/269

CPC classification number: G06T5/005 ,  G06T7/269 ,  G06T2207/10016

Abstract: One or more processing devices access a scene depicting a reference object that includes an annotation identifying a target region to be modified in one or more video frames. The one or more processing devices determine that a target pixel corresponds to a sub-region within the target region that includes hallucinated content. The one or more processing devices determine gradient constraints using gradient values of neighboring pixels in the hallucinated content, the neighboring pixels being adjacent to the target pixel and corresponding to four cardinal directions. The one or more processing devices update color data of the target pixel subject to the determined gradient constraints.

公开(公告)号：US11823357B2

公开(公告)日：2023-11-21

申请号：US17196581

申请日：2021-03-09

Applicant: Adobe Inc.

Inventor： Oliver Wang ,  John Nelson ,  Geoffrey Oxholm ,  Elya Shechtman

IPC: G06T5/00 ,  G06T7/269

CPC classification number: G06T5/005 ,  G06T7/269 ,  G06T2207/10016

Abstract: Certain aspects involve video inpainting in which content is propagated from a user-provided reference video frame to other video frames depicting a scene. One example method includes one or more processing devices that performs operations that include accessing a scene depicting a reference object that includes an annotation identifying a target region to be modified in one or more video frames. The operations also includes computing a target motion of a target pixel that is subject to a motion constraint. The motion constraint is based on a three-dimensional model of the reference object. Further, operations include determining color data of the target pixel to correspond to the target motion. The color data includes a color value and a gradient. Operations also include determining gradient constraints using gradient values of neighbor pixels. Additionally, the processing devices updates the color data of the target pixel subject to the gradient constraints.

公开(公告)号：US20220292649A1

公开(公告)日：2022-09-15

申请号：US17196581

申请日：2021-03-09

Applicant: Adobe Inc.

Inventor： Oliver Wang ,  John Nelson ,  Geoffrey Oxholm ,  Elya Shechtman

IPC: G06T5/00 ,  G06T7/269

Abstract: Certain aspects involve video inpainting in which content is propagated from a user-provided reference video frame to other video frames depicting a scene. One example method includes one or more processing devices that performs operations that include accessing a scene depicting a reference object that includes an annotation identifying a target region to be modified in one or more video frames. The operations also includes computing a target motion of a target pixel that is subject to a motion constraint. The motion constraint is based on a three-dimensional model of the reference object. Further, operations include determining color data of the target pixel to correspond to the target motion. The color data includes a color value and a gradient. Operations also include determining gradient constraints using gradient values of neighbor pixels. Additionally, the processing devices updates the color data of the target pixel subject to the gradient constraints.

公开(公告)号：US11756210B2

公开(公告)日：2023-09-12

申请号：US16817100

申请日：2020-03-12

Applicant: Adobe Inc.

Inventor： Oliver Wang ,  Matthew Fisher ,  John Nelson ,  Geoffrey Oxholm ,  Elya Shechtman ,  Wenqi Xian

IPC: G06V20/40 ,  G06T7/246 ,  G06N20/00 ,  G06V10/44 ,  G06V10/56

CPC classification number: G06T7/246 ,  G06N20/00 ,  G06V10/44 ,  G06V10/56 ,  G06V20/41 ,  G06V20/46

Abstract: Certain aspects involve video inpainting in which content is propagated from a user-provided reference frame to other video frames depicting a scene. For example, a computing system accesses a set of video frames with annotations identifying a target region to be modified. The computing system determines a motion of the target region's boundary across the set of video frames, and also interpolates pixel motion within the target region across the set of video frames. The computing system also inserts, responsive to user input, a reference frame into the set of video frames. The reference frame can include reference color data from a user-specified modification to the target region. The computing system can use the reference color data and the interpolated motion to update color data in the target region across set of video frames.

公开(公告)号：US20210287007A1

公开(公告)日：2021-09-16

申请号：US16817100

申请日：2020-03-12

Applicant: Adobe Inc.

Inventor： Oliver Wang ,  Matthew Fisher ,  John Nelson ,  Geoffrey Oxholm ,  Elya Shechtman ,  Wenqi Xian

IPC: G06K9/00 ,  G06N20/00 ,  G06K9/46

Abstract: Certain aspects involve video inpainting in which content is propagated from a user-provided reference frame to other video frames depicting a scene. For example, a computing system accesses a set of video frames with annotations identifying a target region to be modified. The computing system determines a motion of the target region's boundary across the set of video frames, and also interpolates pixel motion within the target region across the set of video frames. The computing system also inserts, responsive to user input, a reference frame into the set of video frames. The reference frame can include reference color data from a user-specified modification to the target region. The computing system can use the reference color data and the interpolated motion to update color data in the target region across set of video frames.