公开(公告)号：WO2022096101A1

公开(公告)日：2022-05-12

申请号：PCT/EP2020/081132

申请日：2020-11-05

Applicant: HUAWEI TECHNOLOGIES CO., LTD. ,  TULYAKOV, Stepan

Inventor： GEORGOULIS, Stamatios ,  LI, Yuanyou

IPC: G06T3/00

Abstract: The present disclosure relates to the field of image and video processing. In particular, the disclosure is concerned with video interpolation, for instance, for slow motion video applications. Embodiments of the invention propose accordingly a device and a method for video interpolation. The device comprises an image sensor, an event sensor and a processor. The image sensor is configured to obtain a time sequence of frames including a first frame and a second frame obtained after the first frame. The event sensor is configured to obtain a time sequence of events, wherein one or more events are obtained between the first frame and the second frame. The processor is configured to compute an intermediate frame between the first frame and the second frame, based on the first frame, the second frame, and the one or more events obtained between the first frame and the second frame.

公开(公告)号：WO2022096158A1

公开(公告)日：2022-05-12

申请号：PCT/EP2021/057127

申请日：2021-03-19

Applicant: HUAWEI TECHNOLOGIES CO., LTD. ,  UNIVERSITY OF ZURICH

Inventor： TULYAKOV, Stepan ,  GEORGOULIS, Stamatios ,  LI, Yuanyou ,  GEHRIG, Daniel ,  ERBACH, Julius ,  GEHRIG, Mathias ,  SCARAMUZZA, Davide

IPC: G06T3/00

Abstract: The present disclosure relates to the field of image and video processing. In particular, the disclosure is concerned with video interpolation, for instance, for slow motion video applications. Embodiments of the invention propose accordingly a device and a method for video interpolation. The device comprise a image sensor, an event sensor and a processor. The image sensor is configured to obtain a time sequence of frames including a first frame and a second frame obtained after the first frame. The event sensor is configured to obtain a time sequence of events, wherein one or more events are obtained between the first frame and the second frame. The processor is configured to compute an intermediate frame between the first frame and the second frame, based on the first frame, the second frame, and the one or more events obtained between the first frame and the second frame.

公开(公告)号：WO2023083467A1

公开(公告)日：2023-05-19

申请号：PCT/EP2021/081616

申请日：2021-11-15

Applicant: HUAWEI TECHNOLOGIES CO., LTD. ,  UNIVERSITY OF ZURICH

Inventor： TULYAKOV, Stepan ,  BOCCHICCHIO, Alfredo ,  GEORGOULIS, Stamatios ,  LI, Youanyou ,  GEHRIG, Daniel ,  SCARAMUZZA, Davide ,  GEHRIG, Mathias

IPC: G06T7/246 ,  G06V10/82 ,  H04N7/01

Abstract: An image processing apparatus for generating an interpolated frame at a specified time between adjacent image frames of a video. The image processing apparatus includes an input module configured to receive one, two or more key image frames, and a plurality of surrounding events. The image processing apparatus includes a parametric motion model estimator configured to estimate an inter-frame motion based on the key image frames and the plurality of surrounding events, and a warping encoder configured to compute a plurality of multiscale warping interpolation features. The image processing apparatus includes a synthesis encoder configured to compute a plurality of multiscale synthesis interpolation features, and a multiscale feature fusion decoder module configured to generate the interpolated frame. The image processing apparatus enables highly efficient and accurate video frame interpolation in slow-speed motion as well as in high-speed motion.

公开(公告)号：WO2023083466A1

公开(公告)日：2023-05-19

申请号：PCT/EP2021/081609

申请日：2021-11-15

Applicant: HUAWEI TECHNOLOGIES CO., LTD. ,  UNIVERSITY OF ZURICH

Inventor： GEORGOULIS, Stamatios ,  MESSIKOMMER, Nico ,  TULYAKOV, Stepan ,  ERBACH, Julius ,  BOCCHICCHIO, Alfredo ,  GEHRIG, Daniel ,  LI, Yuanyou ,  SCARAMUZZA, Davide

IPC: G06T5/50

Abstract: A HDR imaging device comprising: an image sensor configured to capture a plurality of LDR images; an aligned event sensor with pixels configured to capture a plurality of events between the plurality of LDR images; an image encoder module configured to generate an image feature representation based on each of the LDR images; an event encoder module configured to generate an event feature representation based on a subset of the plurality of events. A feature alignment module configured to: estimate per-pixel offsets between the image feature representation of a specified LDR image and each other LDR image and generate a plurality of aligned feature representations by applying deformable convolutions or warping to the image feature representations based on the estimated offsets. A fusion module configured to generate fused feature representation; and an image decoder module configured to generate a HDR image based on the fused feature representation.