公开(公告)号：US10825221B1

公开(公告)日：2020-11-03

申请号：US16392041

申请日：2019-04-23

Applicant: ADOBE INC.

Inventor： Zhaowen Wang ,  Yipin Zhou ,  Trung Bui ,  Chen Fang

IPC: G06T13/20 ,  G06N3/04 ,  G06N3/08 ,  G06T7/70 ,  H04N5/265 ,  G10L25/30 ,  G06K9/00

Abstract: The present disclosure provides a method for generating a video of a body moving in synchronization with music by applying a first artificial neural network (ANN) to a sequence of samples of an audio waveform of the music to generate a first latent vector describing the waveform and a sequence of coordinates of points of body parts of the body, by applying a first stage of a second ANN to the sequence of coordinates to generate a second latent vector describing movement of the body, by applying a second stage of the second ANN to static images of a person in a plurality of different poses to generate a third latent vector describing an appearance of the person, and by applying a third stage of the second ANN to the first latent vector, the second latent vector, and the third latent vector to generate the video.

公开(公告)号：US20200342646A1

公开(公告)日：2020-10-29

申请号：US16392041

申请日：2019-04-23

Applicant: ADOBE INC.

Inventor： Zhaowen Wang ,  Yipin Zhou ,  Trung Bui ,  Chen Fang

IPC: G06T13/20 ,  G06N3/04 ,  G06N3/08 ,  G06T7/70 ,  G06K9/00 ,  H04N5/265 ,  G10L25/30

Abstract: The present disclosure provides a method for generating a video of a body moving in synchronization with music by applying a first artificial neural network (ANN) to a sequence of samples of an audio waveform of the music to generate a first latent vector describing the waveform and a sequence of coordinates of points of body parts of the body, by applying a first stage of a second ANN to the sequence of coordinates to generate a second latent vector describing movement of the body, by applying a second stage of the second ANN to static images of a person in a plurality of different poses to generate a third latent vector describing an appearance of the person, and by applying a third stage of the second ANN to the first latent vector, the second latent vector, and the third latent vector to generate the video.

公开(公告)号：US10334202B1

公开(公告)日：2019-06-25

申请号：US15907497

申请日：2018-02-28

Applicant: Adobe Inc.

Inventor： Yipin Zhou ,  Zhaowen Wang ,  Chen Fang ,  Trung Huu Bui

IPC: H04N5/60 ,  G06N3/04 ,  G06N3/08

Abstract: Techniques are disclosed for generating audio based on visual information. In some examples, an audio generation system is trained using supervised learning using a training set generated from videos. The trained audio generation system is able to infer audio for provided silent video based on the visual contents of the silent video, and generate raw waveform samples that represent the inferred audio.