公开(公告)号：US09571846B2

公开(公告)日：2017-02-14

申请号：US14039764

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Timothy John Millet ,  Mark P. Rygh ,  Craig M. Okruhlica ,  Jim C. Chou ,  Guy Cote ,  Gaurav S. Gulati ,  Joseph J. Cheng ,  Joseph P. Bratt

IPC: H04N19/423 ,  H04N19/56 ,  H04N19/53

CPC classification number: H04N19/423 ,  H04N19/53

Abstract: Block processing pipeline methods and apparatus in which reference data are stored to a memory according to tile formats to reduce memory accesses when fetching the data from the memory. When the pipeline stores reference data from a current frame being processed to memory as a reference frame, the reference samples are stored in macroblock sequential order. Each macroblock sample set is stored as a tile. Reference data may be stored in tile formats for luma and chroma. Chroma reference data may be stored in tile formats for chroma 4:2:0, 4:2:2, and/or 4:4:4 formats. A stage of the pipeline may write luma and chroma reference data for macroblocks to memory according to one or more of the macroblock tile formats in a modified knight's order. The stage may delay writing the reference data from the macroblocks until the macroblocks have been fully processed by the pipeline.

Abstract translation: 块处理管道方法和装置，其中参考数据根据瓦片格式存储到存储器中，以在从存储器取出数据时减少存储器访问。 当流水线将正在处理的当前帧的参考数据存储为参考帧时，参考样本以宏块顺序存储。 每个宏块样本集被存储为一个图块。 参考数据可以以瓦片和色度的瓦片格式存储。 色度参考数据可以以瓦4：2：0,4：2：2和/或4：4：4格式的瓦片格式存储。 流水线的一个阶段可以根据改进的骑士顺序中的一个或多个宏块瓦片格式将宏块的亮度和色度参考数据写入存储器。 该阶段可以延迟从宏块写入参考数据，直到宏块已被管道完全处理。

公开(公告)号：US09218639B2

公开(公告)日：2015-12-22

申请号：US14039820

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Guy Cote ,  Mark P. Rygh ,  Timothy John Millet ,  Jim C. Chou ,  Joseph J. Cheng

IPC: G06T15/00 ,  G06T1/20 ,  H04N19/61 ,  H04N19/423 ,  H04N19/436

CPC classification number: G06T1/20 ,  H04N19/423 ,  H04N19/436 ,  H04N19/61

Abstract: A knight's order processing method for block processing pipelines in which the next block input to the pipeline is taken from the row below and one or more columns to the left in the frame. The knight's order method may provide spacing between adjacent blocks in the pipeline to facilitate feedback of data from a downstream stage to an upstream stage. The rows of blocks in the input frame may be divided into sets of rows that constrain the knight's order method to maintain locality of neighbor block data. Invalid blocks may be input to the pipeline at the left of the first set of rows and at the right of the last set of rows, and the sets of rows may be treated as if they are horizontally arranged rather than vertically arranged, to maintain continuity of the knight's order algorithm.

Abstract translation: 一种用于块处理管线的骑士订单处理方法，其中从管线的下一个块输入下一个块，并且在该帧中从左侧获取一个或多个列。 骑士的订单方法可以在管道中的相邻块之间提供间隔，以便于数据从下游阶段到上游阶段的反馈。 输入帧中的块行可以被划分为限制骑士命令方法以维持相邻块数据的位置的行的集合。 无效的块可以被输入到第一组行的左侧和最后一组行的右侧的流水线，并且这些行的集合可以被视为水平排列而不是垂直排列，以保持连续性 的骑士秩序算法。

公开(公告)号：US20150092854A1

公开(公告)日：2015-04-02

申请号：US14039729

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： James E. Orr ,  Timothy John Millet ,  Joseph J. Cheng ,  Nitin Bhargava ,  Guy Cote

IPC: H04N19/436 ,  H04N19/433 ,  H04N19/51 ,  H04N19/583 ,  H04N19/43

CPC classification number: H04N19/43 ,  G06T1/20 ,  H04N19/433 ,  H04N19/436 ,  H04N19/51 ,  H04N19/513

Abstract: A block processing pipeline that includes a software pipeline and a hardware pipeline that run in parallel. The software pipeline runs at least one block ahead of the hardware pipeline. The stages of the pipeline may each include a hardware pipeline component that performs one or more operations on a current block at the stage. At least one stage of the pipeline may also include a software pipeline component that determines a configuration for the hardware component at the stage of the pipeline for processing a next block while the hardware component is processing the current block. The software pipeline component may determine the configuration according to information related to the next block obtained from an upstream stage of the pipeline. The software pipeline component may also obtain and use information related to a block that was previously processed at the stage.

Abstract translation: 一个块处理流水线，包括一个软件流水线和并行运行的硬件流水线。 软件管道在硬件管道之前至少运行一个程序段。 流水线的各个阶段可以各自包括在该阶段对当前块执行一个或多个操作的硬件流水线组件。 管道的至少一个阶段还可以包括软件流水线组件，该软件流水线组件在硬件组件正在处理当前块时，在流水线阶段确定用于处理下一个块的硬件组件的配置。 软件管线组件可以根据从流水线的上游级获得的与下一块相关的信息来确定配置。 软件管道组件还可以获得并使用与先前在该阶段处理的块相关的信息。

公开(公告)号：US20150092843A1

公开(公告)日：2015-04-02

申请号：US14039764

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Timothy John Millet ,  Mark P. Rygh ,  Craig M. Okruhlica ,  Jim C. Chou ,  Guy Cote ,  Gaurav S. Gulati ,  Joseph J. Cheng ,  Joseph P. Bratt

IPC: H04N19/57 ,  H04N19/186 ,  H04N19/182

CPC classification number: H04N19/423 ,  H04N19/53

Abstract: Block processing pipeline methods and apparatus in which reference data are stored to a memory according to tile formats to reduce memory accesses when fetching the data from the memory. When the pipeline stores reference data from a current frame being processed to memory as a reference frame, the reference samples are stored in macroblock sequential order. Each macroblock sample set is stored as a tile. Reference data may be stored in tile formats for luma and chroma. Chroma reference data may be stored in tile formats for chroma 4:2:0, 4:2:2, and/or 4:4:4 formats. A stage of the pipeline may write luma and chroma reference data for macroblocks to memory according to one or more of the macroblock tile formats in a modified knight's order. The stage may delay writing the reference data from the macroblocks until the macroblocks have been fully processed by the pipeline.

Abstract translation: 块处理管道方法和装置，其中参考数据根据瓦片格式存储到存储器中，以在从存储器取出数据时减少存储器访问。 当流水线将正在处理的当前帧的参考数据存储为参考帧时，参考样本以宏块顺序存储。 每个宏块样本集被存储为一个图块。 参考数据可以以瓦片和色度的瓦片格式存储。 色度参考数据可以以瓦4：2：0,4：2：2和/或4：4：4格式的瓦片格式存储。 流水线的一个阶段可以根据改进的骑士顺序中的一个或多个宏块瓦片格式将宏块的亮度和色度参考数据写入存储器。 该阶段可以延迟从宏块写入参考数据，直到宏块已被管道完全处理。

公开(公告)号：US20150091921A1

公开(公告)日：2015-04-02

申请号：US14039845

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Guy Cote ,  Timothy John Millet ,  Joseph J. Cheng ,  Mark P. Rygh ,  Jim C. Chou

IPC: G06T1/20

CPC classification number: G06T1/20 ,  G06T1/60 ,  H04N19/176 ,  H04N19/40 ,  H04N19/423 ,  H04N19/436 ,  H04N19/91

Abstract: In the video encoders described herein, blocks of pixels from a video frame may be encoded (e.g., using CAVLC encoding) in a block processing pipeline using wavefront ordering (e.g., in knight's order). Each of the encoded blocks may be written to a particular one of multiple DMA buffers such that the encoded blocks written to each of the buffers represent consecutive blocks of the video frame in scan order. A transcode pipeline may operate in parallel with (or at least overlapping) the operation of the block processing pipeline. The transcode pipeline may read encoded blocks from the buffers in scan order and merge them into a single bit stream (in scan order). A transcoder core of the transcode pipeline may decode the encoded blocks and encode them using a different encoding process (e.g., CABAC). In some cases, the transcoder may be bypassed.

Abstract translation: 在本文描述的视频编码器中，来自视频帧的像素块可以使用波前排序（例如骑士顺序）在块处理流水线中进行编码（例如，使用CAVLC编码）。 每个编码块可以被写入多个DMA缓冲器中的特定一个，使得写入每个缓冲器的编码块以扫描顺序表示视频帧的连续块。 代码流水线可以与（或至少重叠）块处理流水线的操作并行操作。 代码流水线可以以扫描顺序从缓冲器读取编码块，并将它们合并成单个位流（按扫描顺序）。 代码转换流水线的代码转换器核心可以解码编码的块，并使用不同的编码过程（例如，CABAC）对它们进行编码。 在某些情况下，代码转换器可能被旁路。

公开(公告)号：US08924768B2

公开(公告)日：2014-12-30

申请号：US13957998

申请日：2013-08-02

Applicant: Apple Inc.

Inventor： Timothy John Millet ,  Binu K. Mathew ,  Stephan Vincent Schell

IPC: G06F1/04 ,  G06F1/32 ,  H04L12/12

CPC classification number: G06F1/324 ,  G06F1/04 ,  G06F1/3203 ,  G06F1/3209 ,  G06F1/3287 ,  H04L12/12 ,  Y02D10/171 ,  Y02D50/20 ,  Y02D50/40

Abstract: Apparatuses and methods are disclosed for implementing an inter-processor communication channel including power-down functionality. In one embodiment, the apparatus may comprise a first integrated circuit (IC), a second IC coupled to the first IC via a communication interface, wherein the first IC is in one or more low power states and unable to monitor the communication interface. The apparatus may further comprise an inter-processor communication (IPC) channel coupled between the first and second ICs, wherein the IPC channel is separate from the communication interface and wherein the second IC generates at least one advisory signal to the first IC via the IPC channel.

Abstract translation: 公开了用于实现包括断电功能的处理器间通信信道的装置和方法。 在一个实施例中，该装置可以包括第一集成电路（IC），经由通信接口耦合到第一IC的第二IC，其中第一IC处于一个或多个低功率状态并且不能监视通信接口。 该装置还可以包括耦合在第一和第二IC之间的处理器间通信（IPC）信道，其中IPC信道与通信接口分离，并且其中第二IC通过IPC生成至少一个建议信号到第一IC 渠道。

公开(公告)号：US09336558B2

公开(公告)日：2016-05-10

申请号：US14039845

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Guy Cote ,  Timothy John Millet ,  Joseph J. Cheng ,  Mark P. Rygh ,  Jim C. Chou

IPC: H04N19/436 ,  G06T1/20 ,  H04N19/176 ,  G06T1/60 ,  H04N19/423

CPC classification number: G06T1/20 ,  G06T1/60 ,  H04N19/176 ,  H04N19/40 ,  H04N19/423 ,  H04N19/436 ,  H04N19/91

Abstract: In the video encoders described herein, blocks of pixels from a video frame may be encoded (e.g., using CAVLC encoding) in a block processing pipeline using wavefront ordering (e.g., in knight's order). Each of the encoded blocks may be written to a particular one of multiple DMA buffers such that the encoded blocks written to each of the buffers represent consecutive blocks of the video frame in scan order. A transcode pipeline may operate in parallel with (or at least overlapping) the operation of the block processing pipeline. The transcode pipeline may read encoded blocks from the buffers in scan order and merge them into a single bit stream (in scan order). A transcoder core of the transcode pipeline may decode the encoded blocks and encode them using a different encoding process (e.g., CABAC). In some cases, the transcoder may be bypassed.

Abstract translation: 在本文描述的视频编码器中，来自视频帧的像素块可以使用波前排序（例如骑士顺序）在块处理流水线中进行编码（例如，使用CAVLC编码）。 每个编码块可以被写入多个DMA缓冲器中的特定一个，使得写入每个缓冲器的编码块以扫描顺序表示视频帧的连续块。 代码流水线可以与（或至少重叠）块处理流水线的操作并行操作。 代码流水线可以以扫描顺序从缓冲器读取编码块，并将它们合并成单个位流（按扫描顺序）。 代码转换流水线的代码转换器核心可以解码编码的块，并使用不同的编码过程（例如，CABAC）对它们进行编码。 在某些情况下，代码转换器可能被旁路。

公开(公告)号：US09224187B2

公开(公告)日：2015-12-29

申请号：US14039859

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Guy Cote ,  Jim C. Chou ,  Timothy John Millet ,  Manching Ko ,  Weichun Ku

IPC: G06T1/20 ,  G06F13/00 ,  G06T1/60

CPC classification number: B60R3/007 ,  G06T1/20 ,  G06T1/60 ,  H04N19/40 ,  H04N19/433 ,  H04N19/436 ,  H04N19/91

Abstract: Blocks of pixels from a video frame may be encoded in a block processing pipeline using wavefront ordering, e.g. according to knight's order. Each of the encoded blocks may be written to a particular one of multiple buffers such that the blocks written to each of the buffers represent consecutive blocks of the frame in scan order. Stitching information may be written to the buffers at the end of each row. A stitcher may read the rows from the buffers in order and generate a scan order output stream for the frame. The stitcher component may read the stitching information at the end of each row and apply the stitching information to one or more blocks at the beginning of a next row to stitch the next row to the previous row. Stitching may involve modifying pixel(s) of the blocks and/or modifying metadata for the blocks.

Abstract translation: 来自视频帧的像素块可以使用波前排序在块处理流水线中进行编码，例如， 根据骑士的命令。 每个编码块可被写入多个缓冲器中的特定一个缓冲器，使得写入每个缓冲器的块以扫描顺序表示帧的连续块。 拼接信息可以写入每行末尾的缓冲区。 拼接器可以按顺序从缓冲器中读取行，并为该帧生成扫描顺序输出流。 缝合器部件可以读取每行末端的缝合信息，并将缝合信息应用于下一行开始处的一个或多个块，以将下一行缝合到前一行。 缝合可以涉及修改块的像素和/或修改块的元数据。

公开(公告)号：US20150091927A1

公开(公告)日：2015-04-02

申请号：US14039859

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Guy Cote ,  Jim C. Chou ,  Timothy John Millet ,  Manching Ko ,  Weichun Ku

IPC: G06T1/60

CPC classification number: B60R3/007 ,  G06T1/20 ,  G06T1/60 ,  H04N19/40 ,  H04N19/433 ,  H04N19/436 ,  H04N19/91

Abstract: Blocks of pixels from a video frame may be encoded in a block processing pipeline using wavefront ordering, e.g. according to knight's order. Each of the encoded blocks may be written to a particular one of multiple buffers such that the blocks written to each of the buffers represent consecutive blocks of the frame in scan order. Stitching information may be written to the buffers at the end of each row. A stitcher may read the rows from the buffers in order and generate a scan order output stream for the frame. The stitcher component may read the stitching information at the end of each row and apply the stitching information to one or more blocks at the beginning of a next row to stitch the next row to the previous row. Stitching may involve modifying pixel(s) of the blocks and/or modifying metadata for the blocks.

Abstract translation: 来自视频帧的像素块可以使用波前排序在块处理流水线中进行编码，例如， 根据骑士的命令。 每个编码块可被写入多个缓冲器中的特定一个缓冲器，使得写入每个缓冲器的块以扫描顺序表示帧的连续块。 拼接信息可以写入每行末尾的缓冲区。 拼接器可以按顺序从缓冲器中读取行，并为该帧生成扫描顺序输出流。 缝合器部件可以读取每行末端的缝合信息，并将缝合信息应用于下一行开始处的一个或多个块，以将下一行缝合到前一行。 缝合可以涉及修改块的像素和/或修改块的元数据。

公开(公告)号：US20150091920A1

公开(公告)日：2015-04-02

申请号：US14039804

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Mark P. Rygh ,  Guy Cote ,  Timothy John Millet ,  Joseph J. Cheng

IPC: G06T1/20

CPC classification number: G06T1/20 ,  G06T1/60 ,  H04N19/176 ,  H04N19/423 ,  H04N19/436 ,  H04N19/523 ,  H04N19/593

Abstract: Memory latency tolerance methods and apparatus for maintaining an overall level of performance in block processing pipelines that prefetch reference data into a search window. In a general memory latency tolerance method, search window processing in the pipeline may be monitored. If status of search window processing changes in a way that affects pipeline throughput, then pipeline processing may be modified. The modification may be performed according to no stall methods, stall recovery methods, and/or stall prevention methods. In no stall methods, a block may be processed using the data present in the search window without waiting for the missing reference data. In stall recovery methods, the pipeline is allowed to stall, and processing is modified for subsequent blocks to speed up the pipeline and catch up in throughput. In stall prevention methods, processing is adjusted in advance of the pipeline encountering a stall condition.

Abstract translation: 存储器延迟容限方法和装置，用于在预处理参考数据到搜索窗口的块处理管线中维持整体性能水平。 在通用存储器延迟容限方法中，可以监视流水线中的搜索窗口处理。 如果搜索窗口处理的状态以影响流水线吞吐量的方式改变，则可以修改流水线处理。 修改可以根据没有失速方法，失速恢复方法和/或失速预防方法进行。 在没有停止方法的情况下，可以使用搜索窗口中存在的数据来处理块，而不用等待丢失的参考数据。 在失速恢复方法中，允许管道停止，并且修改后续块的处理以加速管道并追赶吞吐量。 在失速预防方法中，在遇到失速状况的管道之前调整处理。