公开(公告)号：US20230403371A1

公开(公告)日：2023-12-14

申请号：US18206459

申请日：2023-06-06

Applicant: SAMSUNG ELECTRONICS CO., LTD.

Inventor： Xin JIN ,  Guotao SHIN ,  Youxin CHEN ,  Longhai WU ,  Jie CHEN ,  Jayoon KOO ,  Cheulhee HAHM

IPC: H04N7/01 ,  G06T7/246

CPC classification number: H04N7/014 ,  G06T7/246 ,  H04N7/0127 ,  G06T2207/10016 ,  G06T2207/20084

Abstract: A method for generating a video intermediate frame, including obtaining a target video frame pair; constructing an image pyramid for each video frame in the target video frame pair; and generating an intermediate frame of the target video frame pair by using a bidirectional optical flow estimation model and a pixel synthesis model in a layer-by-layer recursive calling manner according to an order of the image pyramid from a high layer to a low layer based on the image pyramid, wherein the generating of the intermediate frame of the target video frame pair comprising: repairing a bidirectional optical flow corresponding to a previous layer using the bidirectional optical flow estimation model, and repairing a previous intermediate frame corresponding to the previous layer using the pixel synthesis model.

公开(公告)号：US20230281829A1

公开(公告)日：2023-09-07

申请号：US18168209

申请日：2023-02-13

Applicant: Samsung Electronics Co., Ltd.

Inventor： Xin JIN ,  Guotao SHEN ,  Youxin CHEN ,  Longhai WU ,  Jie CHEN ,  Jayoon KOO

IPC: G06T7/20 ,  G06T3/00 ,  G06T3/40

CPC classification number: G06T7/20 ,  G06T3/0093 ,  G06T3/40 ,  G06T2207/20016 ,  G06T2207/20084

Abstract: A bidirectional optical flow estimation method and apparatus are provided. The method includes acquiring a target image pair of which optical flow is to be estimated, and constructing an image pyramid for each target image in the target image pair respectively, and performing bidirectional optical flow estimation using a pre-trained optical flow estimation model based on the image pyramid, to obtain bidirectional optical flow between the target images. An optical flow estimation module in the optical flow estimation model is recursively called to perform the bidirectional optical flow estimation sequentially based on images of respective layers in the image pyramid according to a preset order, forward warping towards middle processing is performed on an image of a corresponding layer of the image pyramid before each call of the optical flow estimation module, and an image of an intermediate frame obtained by the forward warping towards middle processing is inputted into the optical flow estimation module. With the disclosure, the efficiency and generalization of bidirectional optical flow estimation can be improved, and model training and optical flow estimation overheads can be reduced.

公开(公告)号：US20230237981A1

公开(公告)日：2023-07-27

申请号：US18128743

申请日：2023-03-30

Applicant: SAMSUNG ELECTRONICS CO., LTD.

Inventor： Ziheng YAN ,  Jiang YU ,  Xin JIN ,  Jie CHEN ,  Weiyang SU ,  Youxin CHEN ,  Longhai WU

IPC: G10H1/00

CPC classification number: G10H1/0033 ,  G10H1/0008

Abstract: A method and apparatus for implementing a virtual performance partner are provided. The method includes collecting audio frame data performed by a performer; and for each piece of current audio frame data collected, performing: converting the piece of current audio frame data collected into a current digital score, matching the current digital score with a range of digital scores in a repertoire, and determining a matching digital score in the range of digital scores that matches the current digital score; positioning a position of the matching digital score in the repertoire, and determining a start time of playing a cooperation part of music in a next bar of the matching digital score in the repertoire for a performance partner.

公开(公告)号：US20240428371A1

公开(公告)日：2024-12-26

申请号：US18658694

申请日：2024-05-08

Applicant: SAMSUNG ELECTRONICS CO., LTD.

Inventor： Xin JIN ,  Ban CHEN ,  Longhai Wu ,  Jie CHEN ,  Jayoon KOO ,  Cheulhee HAHM

IPC: G06T3/18 ,  G06V10/74 ,  G06V10/82 ,  G06V20/40

Abstract: A method for generating a video intermediate frame includes performing a warp operation on a plurality of target video frames, based on a bidirectional optical flow between the plurality of target video frames, to obtain a plurality of pictures, determining a similarity between sub-pictures corresponding to image sub-regions in the plurality of pictures, predicting, based on the similarity, a network depth of a frame synthesis network matched with a corresponding image sub-region, the network depth increasing as the similarity decreases, performing synthetic processing, using the frame synthesis network, on corresponding sub-pictures of each of the image sub-regions, based on the network depth matched with the corresponding image sub-region, to obtain a plurality of images, and splicing the plurality of images to obtain intermediate frames of the plurality of target video frames, based on each of the image sub-regions.