公开(公告)号：US20230123820A1

公开(公告)日：2023-04-20

申请号：US17502714

申请日：2021-10-15

Applicant: Adobe Inc.

Inventor： Yangtuanfeng Wang ,  Duygu Ceylan Aksit ,  Krishna Kumar Singh ,  Niloy J Mitra

IPC: G06T13/40 ,  G06T7/73 ,  G06T7/20 ,  G06N3/08

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and method that utilize a character animation neural network informed by motion and pose signatures to generate a digital video through person-specific appearance modeling and motion retargeting. In particular embodiments, the disclosed systems implement a character animation neural network that includes a pose embedding model to encode a pose signature into spatial pose features. The character animation neural network further includes a motion embedding model to encode a motion signature into motion features. In some embodiments, the disclosed systems utilize the motion features to refine per-frame pose features and improve temporal coherency. In certain implementations, the disclosed systems also utilize the motion features to demodulate neural network weights used to generate an image frame of a character in motion based on the refined pose features.

公开(公告)号：US12067659B2

公开(公告)日：2024-08-20

申请号：US17502714

申请日：2021-10-15

Applicant: Adobe Inc.

Inventor： Yangtuanfeng Wang ,  Duygu Ceylan Aksit ,  Krishna Kumar Singh ,  Niloy J Mitra

IPC: G06T13/40 ,  G06N3/045 ,  G06N3/08 ,  G06N3/088 ,  G06T7/20 ,  G06T7/73

CPC classification number: G06T13/40 ,  G06N3/045 ,  G06N3/08 ,  G06N3/088 ,  G06T7/20 ,  G06T7/73 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30196

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and method that utilize a character animation neural network informed by motion and pose signatures to generate a digital video through person-specific appearance modeling and motion retargeting. In particular embodiments, the disclosed systems implement a character animation neural network that includes a pose embedding model to encode a pose signature into spatial pose features. The character animation neural network further includes a motion embedding model to encode a motion signature into motion features. In some embodiments, the disclosed systems utilize the motion features to refine per-frame pose features and improve temporal coherency. In certain implementations, the disclosed systems also utilize the motion features to demodulate neural network weights used to generate an image frame of a character in motion based on the refined pose features.

公开(公告)号：US20230110114A1

公开(公告)日：2023-04-13

申请号：US17499611

申请日：2021-10-12

Applicant: Adobe Inc.

Inventor： Chinthala Pradyumna Reddy ,  Zhifei Zhang ,  Matthew Fisher ,  Hailin Jin ,  Zhaowen Wang ,  Niloy J Mitra

IPC: G06T11/20 ,  G06T3/40

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media for accurately and flexibly generating scalable fonts utilizing multi-implicit neural font representations. For instance, the disclosed systems combine deep learning with differentiable rasterization to generate a multi-implicit neural font representation of a glyph. For example, the disclosed systems utilize an implicit differentiable font neural network to determine a font style code for an input glyph as well as distance values for locations of the glyph to be rendered based on a glyph label and the font style code. Further, the disclosed systems rasterize the distance values utilizing a differentiable rasterization model and combines the rasterized distance values to generate a permutation-invariant version of the glyph corresponding glyph set.

公开(公告)号：US11875435B2

公开(公告)日：2024-01-16

申请号：US17499611

申请日：2021-10-12

Applicant: Adobe Inc.

Inventor： Chinthala Pradyumna Reddy ,  Zhifei Zhang ,  Matthew Fisher ,  Hailin Jin ,  Zhaowen Wang ,  Niloy J Mitra

IPC: G06T11/20 ,  G06T3/40

CPC classification number: G06T11/203 ,  G06T3/40

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media for accurately and flexibly generating scalable fonts utilizing multi-implicit neural font representations. For instance, the disclosed systems combine deep learning with differentiable rasterization to generate a multi-implicit neural font representation of a glyph. For example, the disclosed systems utilize an implicit differentiable font neural network to determine a font style code for an input glyph as well as distance values for locations of the glyph to be rendered based on a glyph label and the font style code. Further, the disclosed systems rasterize the distance values utilizing a differentiable rasterization model and combines the rasterized distance values to generate a permutation-invariant version of the glyph corresponding glyph set.