公开(公告)号：US20240346722A1

公开(公告)日：2024-10-17

申请号：US18423226

申请日：2024-01-25

Applicant: POSTECH RESEARCH AND BUSINESS DEVELOPMENT FOUNDATION

Inventor： Taehyun OH ,  Hyunwoo HA ,  Sungbin KIM

IPC: G06T11/60 ,  G06V10/44

CPC classification number: G06T11/60 ,  G06V10/44

Abstract: There is provided a method for training an image generating model that generates an image from an audio. The method includes selecting at least one frame from a video including a plurality of frames based on a correlation between an audio and an image of each frame; extracting image information and audio information from each of the selected at least one frame; and training an audio feature vector extracting model that extracts an audio feature vector from the audio information, wherein the audio feature vector is aligned within an embedding space with an image feature vector extracted from the image information by a pre-trained image feature vector extracting model.

公开(公告)号：US20250078326A1

公开(公告)日：2025-03-06

申请号：US18441871

申请日：2024-02-14

Applicant: POSTECH RESEARCH AND BUSINESS DEVELOPMENT FOUNDATION

Inventor： ByungKi KWON ,  Taehyun OH ,  Hyeon Woo NAM ,  Ji-Yun KIM

IPC: G06T11/00 ,  G06T3/18 ,  G06V10/54

Abstract: There is provided a method for generating image change data to be performed by an image change data generating apparatus, the method comprising, inputting an initial image and data parameters to a data generator, generating a first image and a second image respectively, within a first time interval by using the data generator based on the initial image and the data parameters, the first image and the second image being in a relationship of consecutive frames with each other, and generating a first image change data by using a pre-trained first proxy network and a second image change data by using a pre-trained second proxy network based on the first image and the second image.