公开(公告)号：US10187655B2

公开(公告)日：2019-01-22

申请号：US14871827

申请日：2015-09-30

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Mark P. Rygh ,  Guy Cõté

IPC: H04N19/53 ,  H04N19/52 ,  H04N19/109 ,  H04N19/567 ,  H04N19/147 ,  H04N19/56 ,  H04N19/179 ,  H04N19/186 ,  H04N19/70 ,  H04N19/117 ,  H04N19/13 ,  H04N19/132 ,  H04N19/40 ,  H04N19/139 ,  H04N19/159 ,  H04N19/593

Abstract: System and method for improving operational efficiency of a video encoding pipeline used to encode image data. In embodiments, the video encoding pipeline includes a low resolution pipeline that includes a low resolution motion estimation block, which generates downscaled image data by reducing resolution of the image data and determines a low resolution inter-frame prediction mode by performing a motion estimation search using the downscaled image data and previously downscaled image data. The video encoding pipeline also includes a main pipeline in parallel with the low resolution pipeline that includes a motion estimation block, which determines a candidate inter-frame prediction mode based at least in part on the low resolution inter-frame prediction mode, and a mode decision block, which determines a first rate-distortion cost associated with the candidate inter-frame prediction mode and determines prediction mode used to prediction encode the image data based at least in part on the first rate-distortion cost.

公开(公告)号：US20170094311A1

公开(公告)日：2017-03-30

申请号：US14871827

申请日：2015-09-30

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Mark P. Rygh ,  Guy Côté

IPC: H04N19/53 ,  H04N19/109 ,  H04N19/567 ,  H04N19/147 ,  H04N19/56 ,  H04N19/179 ,  H04N19/186 ,  H04N19/70 ,  H04N19/117 ,  H04N19/13 ,  H04N19/132 ,  H04N19/40 ,  H04N19/139 ,  H04N19/159 ,  H04N19/593 ,  H04N19/52

CPC classification number: H04N19/53 ,  H04N19/109 ,  H04N19/117 ,  H04N19/13 ,  H04N19/132 ,  H04N19/139 ,  H04N19/147 ,  H04N19/159 ,  H04N19/179 ,  H04N19/186 ,  H04N19/40 ,  H04N19/52 ,  H04N19/56 ,  H04N19/567 ,  H04N19/59 ,  H04N19/593 ,  H04N19/70

Abstract: System and method for improving operational efficiency of a video encoding pipeline used to encode image data. In embodiments, the video encoding pipeline includes a low resolution pipeline that includes a low resolution motion estimation block, which generates downscaled image data by reducing resolution of the image data and determines a low resolution inter-frame prediction mode by performing a motion estimation search using the downscaled image data and previously downscaled image data. The video encoding pipeline also includes a main pipeline in parallel with the low resolution pipeline that includes a motion estimation block, which determines a candidate inter-frame prediction mode based at least in part on the low resolution inter-frame prediction mode, and a mode decision block, which determines a first rate-distortion cost associated with the candidate inter-frame prediction mode and determines prediction mode used to prediction encode the image data based at least in part on the first rate-distortion cost.

公开(公告)号：US20170094304A1

公开(公告)日：2017-03-30

申请号：US14871778

申请日：2015-09-30

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Mark P. Rygh ,  Guy Côté

IPC: H04N19/52 ,  H04N19/573 ,  H04N19/567 ,  H04N19/587 ,  H04N19/583 ,  H04N19/59

CPC classification number: H04N19/52 ,  H04N19/156 ,  H04N19/523 ,  H04N19/533 ,  H04N19/567 ,  H04N19/57 ,  H04N19/573 ,  H04N19/583 ,  H04N19/587 ,  H04N19/59 ,  H04N19/61

Abstract: System and method for improving operational efficiency of a video encoding pipeline used to encode image data. The video encoding pipeline includes a mode decision block, which selects a first inter-frame prediction mode used to prediction encode a first prediction unit, and a motion estimation block, which receives the first inter-frame prediction mode as feedback from the mode decision block when processing a second prediction unit; determines an initial candidate inter-frame prediction mode of the second prediction unit based at least in part on the first inter-frame prediction mode; and determines a final candidate inter-frame prediction mode of the second prediction unit by performing a first motion estimation search based at least in part on the initial candidate inter-frame prediction mode. The mode decision block determines a rate-distortion cost associated with the final candidate inter-frame prediction mode and a prediction mode used to prediction encode the second prediction unit based at least in part on the rate-distortion cost.

公开(公告)号：US09292899B2

公开(公告)日：2016-03-22

申请号：US14037318

申请日：2013-09-25

Applicant: Apple Inc.

Inventor： Marc A. Schaub ,  Joseph J. Cheng ,  Mark P. Rygh ,  Guy Cote

IPC: G06F13/00 ,  G06T1/20 ,  G06T1/60 ,  H04N19/61 ,  H04N19/423 ,  H04N19/436

CPC classification number: G06T1/20 ,  G06T1/60 ,  H04N19/423 ,  H04N19/436 ,  H04N19/61

Abstract: Block processing pipeline methods and apparatus in which pixel data from a reference frame is prefetched into a search window memory. The search window may include two or more overlapping regions of pixels from the reference frame corresponding to blocks from the rows in the input frame that are currently being processed in the pipeline. Thus, the pipeline may process blocks from multiple rows of an input frame using one set of pixel data from a reference frame that is stored in a shared search window memory. The search window may be advanced by one column of blocks by initiating a prefetch for a next column of reference data from a memory. The pipeline may also include a reference data cache that may be used to cache a portion of a reference frame and from which at least a portion of a prefetch for the search window may be satisfied.

Abstract translation: 块处理流水线方法和装置，其中来自参考帧的像素数据被预取到搜索窗口存储器中。 搜索窗口可以包括对应于当前正在流水线处理的输入帧中的行的来自参考帧的两个或更多个重叠区域的像素。 因此，流水线可以使用来自存储在共享搜索窗口存储器中的参考帧的一组像素数据来处理来自输入帧的多行的块。 搜索窗口可以由一列块提前，通过从存储器发起下一列参考数据的预取。 流水线还可以包括可用于缓存参考帧的一部分的参考数据高速缓存，并且可以从该参考数据高速缓冲存储器可以满足搜索窗口的预取的至少一部分。

公开(公告)号：US20150092843A1

公开(公告)日：2015-04-02

申请号：US14039764

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Timothy John Millet ,  Mark P. Rygh ,  Craig M. Okruhlica ,  Jim C. Chou ,  Guy Cote ,  Gaurav S. Gulati ,  Joseph J. Cheng ,  Joseph P. Bratt

IPC: H04N19/57 ,  H04N19/186 ,  H04N19/182

CPC classification number: H04N19/423 ,  H04N19/53

Abstract: Block processing pipeline methods and apparatus in which reference data are stored to a memory according to tile formats to reduce memory accesses when fetching the data from the memory. When the pipeline stores reference data from a current frame being processed to memory as a reference frame, the reference samples are stored in macroblock sequential order. Each macroblock sample set is stored as a tile. Reference data may be stored in tile formats for luma and chroma. Chroma reference data may be stored in tile formats for chroma 4:2:0, 4:2:2, and/or 4:4:4 formats. A stage of the pipeline may write luma and chroma reference data for macroblocks to memory according to one or more of the macroblock tile formats in a modified knight's order. The stage may delay writing the reference data from the macroblocks until the macroblocks have been fully processed by the pipeline.

Abstract translation: 块处理管道方法和装置，其中参考数据根据瓦片格式存储到存储器中，以在从存储器取出数据时减少存储器访问。 当流水线将正在处理的当前帧的参考数据存储为参考帧时，参考样本以宏块顺序存储。 每个宏块样本集被存储为一个图块。 参考数据可以以瓦片和色度的瓦片格式存储。 色度参考数据可以以瓦4：2：0,4：2：2和/或4：4：4格式的瓦片格式存储。 流水线的一个阶段可以根据改进的骑士顺序中的一个或多个宏块瓦片格式将宏块的亮度和色度参考数据写入存储器。 该阶段可以延迟从宏块写入参考数据，直到宏块已被管道完全处理。

公开(公告)号：US20150091921A1

公开(公告)日：2015-04-02

申请号：US14039845

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Guy Cote ,  Timothy John Millet ,  Joseph J. Cheng ,  Mark P. Rygh ,  Jim C. Chou

IPC: G06T1/20

CPC classification number: G06T1/20 ,  G06T1/60 ,  H04N19/176 ,  H04N19/40 ,  H04N19/423 ,  H04N19/436 ,  H04N19/91

Abstract: In the video encoders described herein, blocks of pixels from a video frame may be encoded (e.g., using CAVLC encoding) in a block processing pipeline using wavefront ordering (e.g., in knight's order). Each of the encoded blocks may be written to a particular one of multiple DMA buffers such that the encoded blocks written to each of the buffers represent consecutive blocks of the video frame in scan order. A transcode pipeline may operate in parallel with (or at least overlapping) the operation of the block processing pipeline. The transcode pipeline may read encoded blocks from the buffers in scan order and merge them into a single bit stream (in scan order). A transcoder core of the transcode pipeline may decode the encoded blocks and encode them using a different encoding process (e.g., CABAC). In some cases, the transcoder may be bypassed.

Abstract translation: 在本文描述的视频编码器中，来自视频帧的像素块可以使用波前排序（例如骑士顺序）在块处理流水线中进行编码（例如，使用CAVLC编码）。 每个编码块可以被写入多个DMA缓冲器中的特定一个，使得写入每个缓冲器的编码块以扫描顺序表示视频帧的连续块。 代码流水线可以与（或至少重叠）块处理流水线的操作并行操作。 代码流水线可以以扫描顺序从缓冲器读取编码块，并将它们合并成单个位流（按扫描顺序）。 代码转换流水线的代码转换器核心可以解码编码的块，并使用不同的编码过程（例如，CABAC）对它们进行编码。 在某些情况下，代码转换器可能被旁路。