公开(公告)号：US20250037461A1

公开(公告)日：2025-01-30

申请号：US18361707

申请日：2023-07-28

Applicant: Adobe Inc.

Inventor： Joon-Young LEE ,  Seoung Wug OH ,  John G. NELSON ,  Wujun WANG

IPC: G06V20/40 ,  G06V10/72

Abstract: Embodiments are disclosed for a method including obtaining a region of interest of a current frame of a video sequence depicting an object. The method may further include determining, by a mask propagation model, a likelihood of each pixel of the current frame being associated with the object in the region of interest of the current frame based on the region of interest of the current frame and a fixed number of previous frames of the video sequence including the object. The method may further include replacing a previous frame of the fixed number of previous frames with the current frame. The method may further include displaying the current frame of the video sequence including a masked object in the region of interest of the current frame based on the likelihood of one or more pixels of the current frame being associated with the object.

公开(公告)号：US20230342991A1

公开(公告)日：2023-10-26

申请号：US17726304

申请日：2022-04-21

Applicant: Adobe Inc.

Inventor： Seoung Wug OH ,  Joon-Young LEE ,  Brian PRICE ,  John G. NELSON ,  Wujun WANG ,  Adam PIKIELNY

IPC: G06T11/00 ,  G06T5/00 ,  G06N3/08 ,  G06T7/194

CPC classification number: G06T11/001 ,  G06T5/003 ,  G06N3/084 ,  G06T7/194 ,  G06T5/002 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Embodiments are disclosed for a machine learning-based chroma keying process. The method may include receiving an input including an image depicting a chroma key scene and a color value corresponding to a background color of the image. The method may further include generating a preprocessed image by concatenating the image and the color value. The method may further include providing the preprocessed image to a trained neural network. The method may further include generating, using the trained neural network, an alpha matte representation of the image based on the preprocessed image.

公开(公告)号：US20250029386A1

公开(公告)日：2025-01-23

申请号：US18356892

申请日：2023-07-21

Applicant: Adobe Inc.

Inventor： Joon-Young LEE ,  Seoung Wug OH ,  Ho Kei CHENG ,  Brian PRICE

IPC: G06V20/40 ,  G06T7/174 ,  G06T7/187 ,  G06T7/194 ,  G06V10/77

Abstract: Embodiments are disclosed for performing universal segmentation to mask objects across multiple frames of a video. The method may include determining an image segmentation mask which masks an object of a frame of a video sequence using the frame and an image segmentation module of a segmentation system. The method further includes determining a mask propagation mask which masks the object of the frame of the video sequence using the frame, a representation of a previous frame of the video sequence, and a mask propagation module of the segmentation system. The method further includes determining a frame mask which masks the object of the frame of the video sequence based on a comparison of the image segmentation mask and the mask propagation mask. The method further includes displaying the frame mask of the video sequence.

公开(公告)号：US20240397059A1

公开(公告)日：2024-11-28

申请号：US18322310

申请日：2023-05-23

Applicant: Adobe Inc.

Inventor： Joon-Young LEE ,  Seoung Wug OH ,  Ho Kei CHENG ,  Brian PRICE

IPC: H04N19/172 ,  G06V10/25 ,  H04L9/32

Abstract: A method includes receiving a frame depicting an object. The frame is one frame of a plurality of frames of a video sequence. The method further includes encoding a plurality of tokens of the frame. Each token is a representation of a grid of pixels of the frame. The method further includes selecting a subset of tokens for decoding based on a likelihood of a token satisfying a confidence threshold. The token satisfies the confidence threshold based on a confidence score of the token including a past object in a past frame. The method further includes decoding the subset of tokens using a decoder.

公开(公告)号：US20240005663A1

公开(公告)日：2024-01-04

申请号：US17853671

申请日：2022-06-29

Applicant: Adobe Inc.

Inventor： Joon-Young LEE ,  Seoung Wug OH ,  Sanghyun WOO ,  Kwanyong PARK

IPC: G06V20/40 ,  G06F16/783 ,  G06F16/735 ,  G06T7/11 ,  H04N21/472

CPC classification number: G06V20/49 ,  G06F16/784 ,  G06F16/735 ,  G06T7/11 ,  H04N21/47205 ,  G06T2207/10021

Abstract: Embodiments are disclosed for performing per-clip object segmentation of objects in a video sequence using machine learning. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a query video sequence and memory data, the memory data including a memory video frame from the query video sequence and an annotated memory video frame including an object mask for an object in the memory video frame, segmenting the query video sequence into a plurality of query video clips and passing a first set of query video frames of a first query video clip and the memory data through a trained encoder-decoder network, predicting a modified set of query video frames, including predictions of object masks for the object, and updating the memory data to include one or more frames of the first set of query video frames and the modified set of query video frames.