公开(公告)号：US20150092843A1

公开(公告)日：2015-04-02

申请号：US14039764

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Timothy John Millet ,  Mark P. Rygh ,  Craig M. Okruhlica ,  Jim C. Chou ,  Guy Cote ,  Gaurav S. Gulati ,  Joseph J. Cheng ,  Joseph P. Bratt

IPC: H04N19/57 ,  H04N19/186 ,  H04N19/182

CPC classification number: H04N19/423 ,  H04N19/53

Abstract: Block processing pipeline methods and apparatus in which reference data are stored to a memory according to tile formats to reduce memory accesses when fetching the data from the memory. When the pipeline stores reference data from a current frame being processed to memory as a reference frame, the reference samples are stored in macroblock sequential order. Each macroblock sample set is stored as a tile. Reference data may be stored in tile formats for luma and chroma. Chroma reference data may be stored in tile formats for chroma 4:2:0, 4:2:2, and/or 4:4:4 formats. A stage of the pipeline may write luma and chroma reference data for macroblocks to memory according to one or more of the macroblock tile formats in a modified knight's order. The stage may delay writing the reference data from the macroblocks until the macroblocks have been fully processed by the pipeline.

Abstract translation: 块处理管道方法和装置，其中参考数据根据瓦片格式存储到存储器中，以在从存储器取出数据时减少存储器访问。 当流水线将正在处理的当前帧的参考数据存储为参考帧时，参考样本以宏块顺序存储。 每个宏块样本集被存储为一个图块。 参考数据可以以瓦片和色度的瓦片格式存储。 色度参考数据可以以瓦4：2：0,4：2：2和/或4：4：4格式的瓦片格式存储。 流水线的一个阶段可以根据改进的骑士顺序中的一个或多个宏块瓦片格式将宏块的亮度和色度参考数据写入存储器。 该阶段可以延迟从宏块写入参考数据，直到宏块已被管道完全处理。

公开(公告)号：US20150091921A1

公开(公告)日：2015-04-02

申请号：US14039845

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Guy Cote ,  Timothy John Millet ,  Joseph J. Cheng ,  Mark P. Rygh ,  Jim C. Chou

IPC: G06T1/20

CPC classification number: G06T1/20 ,  G06T1/60 ,  H04N19/176 ,  H04N19/40 ,  H04N19/423 ,  H04N19/436 ,  H04N19/91

Abstract: In the video encoders described herein, blocks of pixels from a video frame may be encoded (e.g., using CAVLC encoding) in a block processing pipeline using wavefront ordering (e.g., in knight's order). Each of the encoded blocks may be written to a particular one of multiple DMA buffers such that the encoded blocks written to each of the buffers represent consecutive blocks of the video frame in scan order. A transcode pipeline may operate in parallel with (or at least overlapping) the operation of the block processing pipeline. The transcode pipeline may read encoded blocks from the buffers in scan order and merge them into a single bit stream (in scan order). A transcoder core of the transcode pipeline may decode the encoded blocks and encode them using a different encoding process (e.g., CABAC). In some cases, the transcoder may be bypassed.

Abstract translation: 在本文描述的视频编码器中，来自视频帧的像素块可以使用波前排序（例如骑士顺序）在块处理流水线中进行编码（例如，使用CAVLC编码）。 每个编码块可以被写入多个DMA缓冲器中的特定一个，使得写入每个缓冲器的编码块以扫描顺序表示视频帧的连续块。 代码流水线可以与（或至少重叠）块处理流水线的操作并行操作。 代码流水线可以以扫描顺序从缓冲器读取编码块，并将它们合并成单个位流（按扫描顺序）。 代码转换流水线的代码转换器核心可以解码编码的块，并使用不同的编码过程（例如，CABAC）对它们进行编码。 在某些情况下，代码转换器可能被旁路。

公开(公告)号：US20170054433A1

公开(公告)日：2017-02-23

申请号：US14831708

申请日：2015-08-20

Applicant: APPLE INC.

Inventor： Tzach Zemer ,  Joseph J. Cheng

IPC: H03H17/06 ,  H04R3/04

CPC classification number: H03H17/0664 ,  H03H17/0226 ,  H04R3/00 ,  H04R3/04 ,  H04R2410/00

Abstract: Systems, apparatuses, and methods for implementing a low power decimator. A decimator may receive a plurality of input samples from a digital microphone. The decimator may include one or more coefficient tables for storing values combining two or more filter coefficients for filtering the received samples. The decimator may utilize a concatenation of multiple samples to perform a lookup of a corresponding coefficient table. The coefficient tables may store only the necessary non-redundant values for all coefficient combinations which can be applied to the multiple samples. The result of the lookup of the coefficient table may have its sign inverted or be zeroed based on the values of the multiple samples.

Abstract translation: 用于实现低功率抽取器的系统，设备和方法。 抽取器可以从数字麦克风接收多个输入样本。 抽取器可以包括一个或多个系数表，用于存储组合用于过滤所接收的样本的两个或更多个滤波器系数的值。 抽取器可以利用多个样本的级联来执行对应系数表的查找。 系数表可以仅存储可应用于多个样本的所有系数组合所需的非冗余值。 系数表查找的结果可以根据多个样本的值反转或归零。

公开(公告)号：US09224186B2

公开(公告)日：2015-12-29

申请号：US14039804

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Mark P. Rygh ,  Guy Cote ,  Timothy John Millet ,  Joseph J. Cheng

IPC: G06T1/20 ,  G06T1/60 ,  H04N19/423 ,  H04N19/436 ,  H04N19/176

CPC classification number: G06T1/20 ,  G06T1/60 ,  H04N19/176 ,  H04N19/423 ,  H04N19/436 ,  H04N19/523 ,  H04N19/593

Abstract: Memory latency tolerance methods and apparatus for maintaining an overall level of performance in block processing pipelines that prefetch reference data into a search window. In a general memory latency tolerance method, search window processing in the pipeline may be monitored. If status of search window processing changes in a way that affects pipeline throughput, then pipeline processing may be modified. The modification may be performed according to no stall methods, stall recovery methods, and/or stall prevention methods. In no stall methods, a block may be processed using the data present in the search window without waiting for the missing reference data. In stall recovery methods, the pipeline is allowed to stall, and processing is modified for subsequent blocks to speed up the pipeline and catch up in throughput. In stall prevention methods, processing is adjusted in advance of the pipeline encountering a stall condition.

Abstract translation: 存储器延迟容限方法和装置，用于在预处理参考数据到搜索窗口的块处理管线中维持整体性能水平。 在通用存储器延迟容限方法中，可以监视流水线中的搜索窗口处理。 如果搜索窗口处理的状态以影响流水线吞吐量的方式改变，则可以修改流水线处理。 修改可以根据没有失速方法，失速恢复方法和/或失速预防方法进行。 在没有停止方法的情况下，可以使用搜索窗口中存在的数据来处理块，而不用等待丢失的参考数据。 在失速恢复方法中，允许管道停止，并且修改后续块的处理以加速管道并追赶吞吐量。 在失速预防方法中，在遇到失速状况的管道之前调整处理。

公开(公告)号：US09215472B2

公开(公告)日：2015-12-15

申请号：US14039729

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： James E. Orr ,  Timothy John Millet ,  Joseph J. Cheng ,  Nitin Bhargava ,  Guy Cote

IPC: G06T1/20 ,  G06T1/00 ,  G06F15/00 ,  H04N19/43 ,  H04N19/433 ,  H04N19/436 ,  H04N19/51 ,  H04N19/583 ,  H04N19/513

CPC classification number: H04N19/43 ,  G06T1/20 ,  H04N19/433 ,  H04N19/436 ,  H04N19/51 ,  H04N19/513

Abstract: A block processing pipeline that includes a software pipeline and a hardware pipeline that run in parallel. The software pipeline runs at least one block ahead of the hardware pipeline. The stages of the pipeline may each include a hardware pipeline component that performs one or more operations on a current block at the stage. At least one stage of the pipeline may also include a software pipeline component that determines a configuration for the hardware component at the stage of the pipeline for processing a next block while the hardware component is processing the current block. The software pipeline component may determine the configuration according to information related to the next block obtained from an upstream stage of the pipeline. The software pipeline component may also obtain and use information related to a block that was previously processed at the stage.

Abstract translation: 一个块处理流水线，包括一个软件流水线和并行运行的硬件流水线。 软件管道在硬件管道之前至少运行一个程序段。 流水线的各个阶段可以各自包括在该阶段对当前块执行一个或多个操作的硬件流水线组件。 管道的至少一个阶段还可以包括软件流水线组件，该软件流水线组件在硬件组件正在处理当前块时，在流水线阶段确定用于处理下一个块的硬件组件的配置。 软件管线组件可以根据从流水线的上游级获得的与下一块相关的信息来确定配置。 软件管道组件还可以获得并使用与先前在该阶段处理的块相关的信息。

公开(公告)号：US20150091914A1

公开(公告)日：2015-04-02

申请号：US14039820

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Guy Cote ,  Mark P. Rygh ,  Timothy John Millet ,  Jim C. Chou ,  Joseph J. Cheng

IPC: G06T1/20

CPC classification number: G06T1/20 ,  H04N19/423 ,  H04N19/436 ,  H04N19/61

Abstract: A knight's order processing method for block processing pipelines in which the next block input to the pipeline is taken from the row below and one or more columns to the left in the frame. The knight's order method may provide spacing between adjacent blocks in the pipeline to facilitate feedback of data from a downstream stage to an upstream stage. The rows of blocks in the input frame may be divided into sets of rows that constrain the knight's order method to maintain locality of neighbor block data. Invalid blocks may be input to the pipeline at the left of the first set of rows and at the right of the last set of rows, and the sets of rows may be treated as if they are horizontally arranged rather than vertically arranged, to maintain continuity of the knight's order algorithm.

Abstract translation: 一种用于块处理管线的骑士订单处理方法，其中从管线的下一个块输入下一个块，并且在该帧中从左侧获取一个或多个列。 骑士的订单方法可以在管道中的相邻块之间提供间隔，以便于数据从下游阶段到上游阶段的反馈。 输入帧中的块行可以被划分为限制骑士命令方法以维持相邻块数据的位置的行的集合。 无效的块可以被输入到第一组行的左侧和最后一组行的右侧的流水线，并且这些行的集合可以被视为水平排列而不是垂直排列，以保持连续性 的骑士秩序算法。

公开(公告)号：US20150084970A1

公开(公告)日：2015-03-26

申请号：US14037318

申请日：2013-09-25

Applicant: Apple Inc.

Inventor： Marc A. Schaub ,  Joseph J. Cheng ,  Mark P. Rygh ,  Guy Cote

IPC: G06T1/20 ,  G06T1/60

CPC classification number: G06T1/20 ,  G06T1/60 ,  H04N19/423 ,  H04N19/436 ,  H04N19/61

Abstract: Block processing pipeline methods and apparatus in which pixel data from a reference frame is prefetched into a search window memory. The search window may include two or more overlapping regions of pixels from the reference frame corresponding to blocks from the rows in the input frame that are currently being processed in the pipeline. Thus, the pipeline may process blocks from multiple rows of an input frame using one set of pixel data from a reference frame that is stored in a shared search window memory. The search window may be advanced by one column of blocks by initiating a prefetch for a next column of reference data from a memory. The pipeline may also include a reference data cache that may be used to cache a portion of a reference frame and from which at least a portion of a prefetch for the search window may be satisfied.

Abstract translation: 块处理流水线方法和装置，其中来自参考帧的像素数据被预取到搜索窗口存储器中。 搜索窗口可以包括对应于当前正在流水线处理的输入帧中的行的来自参考帧的两个或更多个重叠区域的像素。 因此，流水线可以使用来自存储在共享搜索窗口存储器中的参考帧的一组像素数据来处理来自输入帧的多行的块。 搜索窗口可以由一列块提前，通过从存储器发起下一列参考数据的预取。 流水线还可以包括可用于缓存参考帧的一部分的参考数据高速缓存，并且可以从该参考数据高速缓冲存储器可以满足搜索窗口的预取的至少一部分。

公开(公告)号：US09762919B2

公开(公告)日：2017-09-12

申请号：US14472119

申请日：2014-08-28

Applicant: Apple Inc.

Inventor： Guy Cote ,  Joseph P. Bratt ,  Timothy J. Millet ,  Shing I. Kong ,  Joseph J. Cheng

IPC: H04N19/127 ,  H04N19/176 ,  H04N19/186 ,  H04N19/423 ,  H04N19/433 ,  G06F12/00 ,  H04N19/42 ,  H04N19/172 ,  G06T1/60

CPC classification number: H04N19/186 ,  G06F12/00 ,  G06F12/0207 ,  G06F12/0862 ,  G06F12/121 ,  G06F2212/1024 ,  G06F2212/455 ,  G06F2212/6024 ,  G06F2212/6026 ,  G06T1/60 ,  H04N19/127 ,  H04N19/172 ,  H04N19/176 ,  H04N19/423 ,  H04N19/433 ,  H04N19/439

Abstract: Methods and apparatus for caching reference data in a block processing pipeline. A cache may be implemented to which reference data corresponding to motion vectors for blocks being processed in the pipeline may be prefetched from memory. Prefetches for the motion vectors may be initiated one or more stages prior to a processing stage. Cache tags for the cache may be defined by the motion vectors. When a motion vector is received, the tags can be checked to determine if there are cache block(s) corresponding to the vector (cache hits) in the cache. Upon a cache miss, a cache block in the cache is selected according to a replacement policy, the respective tag is updated, and a prefetch (e.g., via DMA) for the respective reference data is issued.

公开(公告)号：US09641158B2

公开(公告)日：2017-05-02

申请号：US14831708

申请日：2015-08-20

Applicant: Apple Inc.

Inventor： Tzach Zemer ,  Joseph J. Cheng

IPC: H04R3/04 ,  H03G5/00 ,  H03H17/06

CPC classification number: H03H17/0664 ,  H03H17/0226 ,  H04R3/00 ,  H04R3/04 ,  H04R2410/00

Abstract: Systems, apparatuses, and methods for implementing a low power decimator. A decimator may receive a plurality of input samples from a digital microphone. The decimator may include one or more coefficient tables for storing values combining two or more filter coefficients for filtering the received samples. The decimator may utilize a concatenation of multiple samples to perform a lookup of a corresponding coefficient table. The coefficient tables may store only the necessary non-redundant values for all coefficient combinations which can be applied to the multiple samples. The result of the lookup of the coefficient table may have its sign inverted or be zeroed based on the values of the multiple samples.

公开(公告)号：US20170052782A1

公开(公告)日：2017-02-23

申请号：US14831955

申请日：2015-08-21

Applicant: Apple Inc.

Inventor： Binu K. Mathew ,  Julia C. Erhard ,  Joseph J. Cheng

IPC: G06F9/30

CPC classification number: G06F9/325 ,  G06F8/443 ,  G06F9/30065

Abstract: An apparatus may include a counter circuit and an execution unit. The execution unit may be configured to receive and execute a first instruction. The first instruction may include a first number corresponding to a first number of instructions of a plurality of instructions, a second number corresponding to a number of times to execute a subset of the plurality of instructions, and a third number corresponding to a number of instructions in the subset. The execution unit may be further configured to initialize a first count value in the counter circuit to the second number in response to the execution of the first instruction, to execute the first number of the plurality of instructions, and to execute the subset of the plurality of instructions. The counter circuit may be configured to modify the first count value in response to determining a last instruction of the subset has been retired.

Abstract translation: 装置可以包括计数器电路和执行单元。 执行单元可以被配置为接收和执行第一指令。 第一指令可以包括对应于多个指令的第一数目的指令的第一数字，对应于执行多个指令的子集的次数的第二数字，以及对应于多个指令的第三个数字 在子集中。 执行单元还可以被配置为响应于第一指令的执行而将计数器电路中的第一计数值初始化为第二数量，以执行多个指令的第一数目，并执行多个指令的子集 的指示。 计数器电路可以被配置为响应于确定已经退出的子集的最后指令来修改第一计数值。