公开(公告)号：US20240340438A1

公开(公告)日：2024-10-10

申请号：US18593281

申请日：2024-03-01

Applicant: Apple Inc.

Inventor： Yeqing WU ,  Yunfei ZHENG ,  Yixin DU ,  Alican NALCI ,  Hilmi E. EGILMEZ ,  Guoxin JIN ,  Alexandros TOURAPIS ,  Jun XIN ,  Hsi-Jung WU

IPC: H04N19/52 ,  H04N19/105 ,  H04N19/124 ,  H04N19/127 ,  H04N19/154 ,  H04N19/172 ,  H04N19/176

CPC classification number: H04N19/52 ,  H04N19/105 ,  H04N19/124 ,  H04N19/127 ,  H04N19/154 ,  H04N19/172 ,  H04N19/176

Abstract: Techniques are proposed to improve temporal motion projection in video coding. Candidate reference frames available for use in temporal motion projection are sorted in processing order according to scores assigned based on estimates of the reference frames' suitability for prediction. Such estimates may be based on temporal distance between each candidate reference frame and that reference frame's prediction references. Estimates may be based, for each reference frame, based on an estimate of coding quality of a reference frame from which the respective candidate reference frame makes a prediction reference. Once sorted, the candidate reference frames may be processing in the sorting order to supply prediction data to a current frame that is to be coded from the candidate reference frames. Additionally, hardware friendly designs of motion field hole filling and motion vector smoothing operations are proposed. Such designs can reduce hardware implementation complexity and benefit hardware parallel processing in several aspects: by removing the dependency among different processing block rows for hole filling and motion vector smoothing so that it is becomes easier and friendlier to achieve hardware parallel processing; by reducing the hardware bandwidth loading overhead; by improving hardware pipeline throughput; and/or by avoiding adding a line buffer to store the data from the above row since a line buffer will increase hardware cost.

公开(公告)号：US20230096567A1

公开(公告)日：2023-03-30

申请号：US17951919

申请日：2022-09-23

Applicant: APPLE INC.

Inventor： Alican NALCI ,  Alexandros TOURAPIS ,  Hsi-Jung WU ,  Jiefu ZHAI ,  Jingteng XUE ,  Jun XIN ,  Mei GUO ,  Xingyu ZHANG ,  Yeqing WU ,  Yunfei ZHENG ,  Jean Begaint

IPC: H04N19/147 ,  H04N19/42 ,  H04N19/172 ,  H04N19/176 ,  H04N19/186 ,  H04N19/119 ,  H04N19/91 ,  H04N19/70 ,  H04N19/124 ,  H04N19/60

Abstract: Improved neural-network-based image and video coding techniques are presented, including hybrid techniques that include both tools of a host codec and neural-network-based tools. In these improved techniques, the host coding tools may include conventional video coding standards such H.266 (VVC). In an aspects, source frames may be partitioned and either host or neural-network-based tools may be selected per partition. Coding parameter decisions for a partition may be constrained based on the partitioning and coding tool selection. Rate control for host and neural network tools may be combined. Multi-stage processing of neural network output may use a checkerboard prediction pattern.

公开(公告)号：US20240129472A1

公开(公告)日：2024-04-18

申请号：US18468900

申请日：2023-09-18

Applicant: Apple Inc.

Inventor： Yeqing WU ,  Yunfei ZHENG ,  Alican NALCI ,  Yixin DU ,  Hilmi Enes EGILMEZ ,  Guoxin JIN ,  Alexandros TOURAPIS ,  Jun XIN ,  Hsi-Jung WU

IPC: H04N19/13 ,  H04N19/129 ,  H04N19/176

CPC classification number: H04N19/13 ,  H04N19/129 ,  H04N19/176

Abstract: Improved lossless entropy coding techniques for coding of image data include selecting a context for entropy coding based on an ordered scan path of possible context locations. A symbol for a current location within a source image may be entropy coded based on a context of prior encoded symbols of other locations within source images, where the context is selected based on an ordered scan path enumerating a series of potential context locations within one or more source images. To select a context, a predetermined number of prior symbols may be selected by qualifying or disqualifying locations in the scan path, and then the current symbol may be encoded with a context based on prior symbols corresponding to the first qualifying context locations in the order of the scan path.

公开(公告)号：US20230143147A1

公开(公告)日：2023-05-11

申请号：US17979027

申请日：2022-11-02

Applicant: Apple Inc.

Inventor： Yixin DU ,  Alexandros TOURAPIS ,  Yunfei ZHENG ,  Jun XIN ,  Alican NALCI ,  Mei T. GUO ,  Yeqing WU ,  Hsi-Jung WU

IPC: H04N19/82 ,  H04N19/186 ,  H04N19/70

CPC classification number: H04N19/82 ,  H04N19/70 ,  H04N19/186

Abstract: A cross-component based filtering system is disclosed for video coders and decoders. The filtering system may include a filter having an input for a filter offset and an input for samples reconstructed from coded video data representing a native component of source video on which the filter operates. The offset may be generated at least in part from a sample classifier that classifies samples reconstructed from coded video data representing a color component of the source video orthogonal to the native component according to sample intensity.

公开(公告)号：US20220191473A1

公开(公告)日：2022-06-16

申请号：US17568266

申请日：2022-01-04

Applicant: Apple Inc.

Inventor： Jiefu ZHAI ,  Xingyu ZHANG ,  Xiaosong ZHOU ,  Jun XIN ,  Hsi-Jung WU ,  Yeping SU

IPC: H04N19/105 ,  H04N19/147 ,  H04N19/159 ,  H04N19/176 ,  H04N19/61 ,  G06N3/08

Abstract: Systems and methods disclosed for video compression, utilizing neural networks for predictive video coding. Processes employed combine multiple banks of neural networks with codec system components to carry out the coding and decoding of video data.

公开(公告)号：US20250080759A1

公开(公告)日：2025-03-06

申请号：US18794601

申请日：2024-08-05

Applicant: APPLE INC.

Inventor： Guoxin JIN ,  Yeqing WU ,  Yunfei ZHENG ,  Dazhong ZHANG ,  Dzung T. HOANG ,  Felix C. FERNANDES ,  Van Luong PHAM ,  Jiancong LUO ,  Yixin DU ,  Hilmi Enes EGILMEZ ,  Alican NALCI ,  Jun XIN ,  Hsi-Jung WU

IPC: H04N19/176 ,  H04N19/117 ,  H04N19/159 ,  H04N19/46 ,  H04N19/70 ,  H04N19/82

Abstract: Techniques are disclosed for coding video in applications where regions of video are inactive on a frame to frame basis. According to the techniques, coding processes update and reconstruct only a subset of pixel blocks of pixels within a frame, while other pixel blocks are retained from a previously coded frame stored in a coder's or decoder's reference frame buffer. The technique is called Backward Reference Updating (or “BRU”) for convenience. At a desired pixel block granularity, based on the activity between a current frame to be coded and its reference frame(s), BRU will only perform prediction, transform, quantization, and reconstruction on selected regions that are determined to be active. The reconstructed pixels in these active regions are directly placed onto a specified reference frame in memory instead of creating a new frame. Therefore, fewer memory transfers need to be performed.

公开(公告)号：US20240073438A1

公开(公告)日：2024-02-29

申请号：US18452194

申请日：2023-08-18

Applicant: Apple Inc.

Inventor： Yeqing WU ,  Yunfei ZHENG ,  Guoxin JIN ,  Yixin DU ,  Alican NALCI ,  Hilmi Enes EGILMEZ ,  Jun XIN ,  Hsi-Jung WU

IPC: H04N19/513 ,  H04N19/176

CPC classification number: H04N19/513 ,  H04N19/176

Abstract: Techniques are disclosed for improved video coding with virtual reference frames. A motion vector for prediction of a pixel block from a reference may be constrained based on the reference. In as aspect, if the reference is a temporally interpolated virtual reference frame with corresponding time close to the time of the current pixel block, the motion vector for prediction may be constrained magnitude and/or precision. In another aspect, a bitstream syntax for encoding the constrained motion vector may also be constrained. In this manner, the techniques proposed herein contribute to improved coding efficiencies.

公开(公告)号：US20230300341A1

公开(公告)日：2023-09-21

申请号：US18157360

申请日：2023-01-20

Applicant: APPLE INC.

Inventor： Yeqing WU ,  Yunfei ZHENG ,  Alexandros TOURAPIS ,  Alican NALCI ,  Yixin DU ,  Hilmi Enes EGILMEZ ,  Albert E. KEINATH ,  Jun XIN ,  Hsi-Jung WU

IPC: H04N19/139 ,  H04N19/176

CPC classification number: H04N19/139 ,  H04N19/176

Abstract: Techniques are disclosed for generating virtual reference frames that may be used for prediction of input video frames. The virtual reference frames may be derived from already-coded reference frames and thereby incur reduced signaling overhead. Moreover, signaling of virtual reference frames may be avoided until an encoder selects the virtual reference frame as a prediction reference for a current frame. In this manner, the techniques proposed herein contribute to improved coding efficiencies.

公开(公告)号：US20230142771A1

公开(公告)日：2023-05-11

申请号：US17979025

申请日：2022-11-02

Applicant: Apple Inc.

Inventor： Yixin DU ,  Alexandros TOURAPIS ,  Yunfei ZHENG ,  Jun XIN ,  Mukta S. Gore ,  Alican NALCI ,  Mei T. GUO ,  Yeqing WU ,  Hsi-Jung WU

IPC: H04N19/82 ,  H04N19/132 ,  H04N19/186 ,  H04N19/176

CPC classification number: H04N19/82 ,  H04N19/132 ,  H04N19/186 ,  H04N19/176

Abstract: A filtering system for video coders and decoders is disclosed that includes a feature detector having an input for samples reconstructed from coded video data representing a color component of source video, and having an output for data identifying a feature recognized therefrom, an offset calculator having an input for the feature identification data from the feature detector and having an output for a filter offset, and a filter having an input for the filter offset from the offset calculator and an input for the reconstructed samples, and having an output for filtered samples. The filtering system is expected to improve operations of video coder/decoder filtering systems by selecting filtering offsets from analysis of recovered video data in a common color plane as the samples that will be filtered.

公开(公告)号：US20200236349A1

公开(公告)日：2020-07-23

申请号：US16254528

申请日：2019-01-22

Applicant: Apple Inc.

Inventor： Jiefu ZHAI ,  Xingyu ZHANG ,  Xiaosong ZHOU ,  Jun XIN ,  Hsi-Jung WU ,  Yeping SU

IPC: H04N19/105 ,  H04N19/147 ,  H04N19/159 ,  H04N19/176 ,  H04N19/61 ,  G06N3/08

Abstract: Systems and methods disclosed for video compression, utilizing neural networks for predictive video coding. Processes employed combine multiple banks of neural networks with codec system components to carry out the coding and decoding of video data.