公开(公告)号：US20230196817A1

公开(公告)日：2023-06-22

申请号：US17552857

申请日：2021-12-16

Applicant: Adobe Inc.

Inventor： Seoung Wug Oh ,  Miran Heo ,  Joon-Young Lee

IPC: G06V40/10 ,  G06T7/10 ,  G06T7/70 ,  G06V10/40

CPC classification number: G06V40/103 ,  G06T7/10 ,  G06T7/70 ,  G06V10/40 ,  G06T2207/20084 ,  G06T2207/30196

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generate joint-based segmentation masks for digital objects portrayed in digital videos. In particular, in one or more embodiments, the disclosed systems utilize a video masking model having a pose tracking neural network and a segmentation neural network to generate the joint-based segmentation masks. To illustrate, in some embodiments, the disclosed systems utilize the pose tracking neural network to identify a set of joints of the digital object across the frames of the digital video. The disclosed systems further utilize the segmentation neural network to generate joint-based segmentation masks for the video frames that portray the object using the identified joints. In some cases, the segmentation neural network includes a multi-layer perceptron mixer layer for mixing visual features propagated via convolutional layers.