Hierarchical grid interpolation systems and methods

    公开(公告)号:US12106444B2

    公开(公告)日:2024-10-01

    申请号:US17356216

    申请日:2021-06-23

    申请人: Apple Inc.

    发明人: Jim C. Chou Jian Zhou

    IPC分类号: G06T3/18 G06T3/04 G06T3/4007

    CPC分类号: G06T3/18 G06T3/04 G06T3/4007

    摘要: An electronic device may include an electronic display to display an image based on processed image data. The electronic device may also include image processing circuitry to determine a hierarchical grid having multiple grid points divided into grid partitions. A first set of grid points associated with a first set of grid partitions may include a first set of mappings to corresponding coordinates of input image data in a source frame. The image processing circuitry may also interpolate between the first set of grid points to determine a second set of grid points of having a second set of mappings to corresponding coordinates of the input image data based on the first set of mappings. The image processing circuitry may also generate the processed image data by applying the first set of mappings and the second set of mappings to the input image data.

    METHODS AND APPARATUS FOR FRAME INTERPOLATION WITH OCCLUDED MOTION

    公开(公告)号:US20240320786A1

    公开(公告)日:2024-09-26

    申请号:US18358293

    申请日:2023-07-25

    申请人: GoPro, Inc.

    发明人: Robert McIntosh

    摘要: Systems, apparatus, and methods adding post-processing motion blur to video and/or frame interpolation with occluded motion. Conventional post-processing techniques relied on the filmmaker to select and stage their shots. Different motion blur techniques were designed to fix certain types of footage. Vector blur is one technique that “smears” pixel information in the direction of movement. Frame interpolation and stacking attempts to create motion blur by stacking interpolated frames together. Each technique has its own set of limitations. Various embodiments use a combination of motion blur techniques in post-processing for better, more realistic outcomes with faster/more efficient rendering times. In some cases, this may enable adaptive quality post-processing that may be performed in mobile/embedded ecosystems. Various embodiments use a combination of video frame interpolation techniques for better interpolated frames with faster/more efficient rendering times.

    Frame Interpolation Using Both Optical Motion And In-Game Motion

    公开(公告)号:US20240311959A1

    公开(公告)日:2024-09-19

    申请号:US18605789

    申请日:2024-03-14

    申请人: MediaTek Inc.

    IPC分类号: G06T3/4007 G06T7/246

    CPC分类号: G06T3/4007 G06T7/248

    摘要: A frame interpolation method generates an interpolated frame that is temporally between a first frame and a second frame. A first and a second interpolated frames are generated using motion vectors from a first motion estimator and a second motion estimator, respectively. A weighting map is generated based on indications from the first motion estimator. First pixel locations and second pixel locations in the weighting map are assigned weight values of 1 and 0, respectively. A weighted combination is calculated using the weighting map to produce the interpolated frame output, which includes the first pixel locations from the first interpolated frame and the second pixel locations from the second interpolated frame. The first and the second motion estimators may be an optical flow estimator and the game engine renderer, respectively. Alternatively, the first and the second motion estimators may be the game engine renderer and the optical flow estimator, respectively.

    Systems and methods for spline-based object tracking

    公开(公告)号:US12094078B2

    公开(公告)日:2024-09-17

    申请号:US17665357

    申请日:2022-02-04

    申请人: Netflix, Inc.

    摘要: The disclosed computer-implemented method may include (1) accessing a video portraying an object within a set of frames, (2) defining a subset of key frames within the video based on movement of the object across the set of frames, (3) generating, for each key frame within the subset of key frames, a spline outlining the object within the key frame, (4) receiving input to adjust, for a selected key frame within the subset of key frames, a corresponding spline, and (5) interpolating the adjusted spline with a spline in a sequentially proximate key frame to define the object in frames between the selected key frame and the sequentially proximate key frame. Various other methods, systems, and computer-readable media are also disclosed.

    Method to improve accuracy of quantized multi-stage object detection network

    公开(公告)号:US12073599B2

    公开(公告)日:2024-08-27

    申请号:US18115188

    申请日:2023-02-28

    发明人: Zhao-Qi Fang

    摘要: An apparatus includes a memory and a processor. The memory may be configured to store image data of an input image. The processor may be configured to detect one or more objects in the input image using a quantized multi-stage object detection network, where quantization of the quantized multi-stage object detection network includes (i) generating quantized image data by performing a first data range analysis on the image data of the input image, (ii) generating a feature map and proposal bounding boxes by applying a region proposal network (RPN) to the quantized image data, (iii) performing a region of interest pooling operation on the feature map and a plurality of ground truth boxes corresponding to the proposal bounding boxes generated by the RPN, (iv) generating quantized region of interest pooling results by performing a second data range analysis on results from the region of interest pooling operation, and (v) applying a region-based convolutional neural network (RCNN) to the quantized region of interest pooling results.