公开(公告)号：US20230169316A1

公开(公告)日：2023-06-01

申请号：US17538138

申请日：2021-11-30

Applicant: Apple Inc.

Inventor： Christopher L. Mills

IPC: G06N3/063 ,  G06F7/78

CPC classification number: G06N3/063 ,  G06F7/78

Abstract: Embodiments of the present disclosure relate to indexing in a neural processor circuit. The neural processor circuit includes multiple neural engine circuits and a data processor circuit directly coupled to at least one of the neural engine circuits. The at least one neural engine circuit performs a convolution operation on input data to generate output data. The data processor circuit includes a buffer memory and an indexing circuit coupled to the buffer memory. The buffer memory stores an index tensor and the output data as a source tensor. The indexing circuit fetches a portion of the source tensor from the buffer memory by referencing the index tensor representing indexing information into the portion of the source tensor.

公开(公告)号：US20230099652A1

公开(公告)日：2023-03-30

申请号：US17991373

申请日：2022-11-21

Applicant: Apple Inc.

Inventor： Erik Norden ,  Liran Fishel ,  Sung Hee Park ,  Jaewon Shin ,  Christopher L. Mills ,  Seungjin Lee ,  Fernando A. Mujica

IPC: G06N3/04 ,  G06F1/3296 ,  G06N3/08

Abstract: Embodiments relate to a neural processor circuit with scalable architecture for instantiating one or more neural networks. The neural processor circuit includes a data buffer coupled to a memory external to the neural processor circuit, and a plurality of neural engine circuits. To execute tasks that instantiate the neural networks, each neural engine circuit generates output data using input data and kernel coefficients. A neural processor circuit may include multiple neural engine circuits that are selectively activated or deactivated according to configuration data of the tasks. Furthermore, an electronic device may include multiple neural processor circuits that are selectively activated or deactivated to execute the tasks.

公开(公告)号：US20230081023A1

公开(公告)日：2023-03-16

申请号：US17989275

申请日：2022-11-17

Applicant: Apple Inc.

Inventor： Christopher L. Mills ,  Kenneth W. Waters

IPC: G06N3/063 ,  G06N5/04 ,  G06N3/02 ,  G06F9/38 ,  G06F9/48 ,  G06N3/08

Abstract: Embodiments relate to a neural processor circuit including one or more planar engine circuits that perform non-convolution operations in parallel with convolution operations performed by one or more neural engine circuits. The neural engine circuits perform the convolution operations on neural input data corresponding to one or more neural engine tasks to generate neural output data. The planar engine circuits perform non-convolution operations on planar input data corresponding to one or more planar engine tasks to generate planar output data. A data processor circuit in the neural processor circuit addresses data dependency between the one or more neural engine tasks and the one or more planar engine tasks by controlling reading of the neural output data as the planar input data by the planar engine circuits or reading of the planar output data as the neural input data by the neural engine circuits.

公开(公告)号：US11537864B2

公开(公告)日：2022-12-27

申请号：US16695782

申请日：2019-11-26

Applicant: Apple Inc.

Inventor： Christopher L. Mills ,  Kenneth W. Waters ,  Youchang Kim

IPC: G06G7/00 ,  G06N3/00 ,  G06N3/063 ,  G06F9/30 ,  G06F9/54

Abstract: Embodiments relate to a neural processor that includes one or more neural engine circuits and planar engine circuits. The neural engine circuits can perform convolution operations of input data with one or more kernels to generate outputs. The planar engine circuit is coupled to the plurality of neural engine circuits. A planar engine circuit can be configured to multiple modes. In a reduction mode, the planar engine circuit may process values arranged in one or more dimensions of input to generate a reduced value. The reduced values across multiple input data may be accumulated. The planar engine circuit may program a filter circuit as a reduction tree to gradually reduce the data into a reduced value. The reduction operation reduces the size of one or more dimensions of a tensor.

公开(公告)号：US20220398440A1

公开(公告)日：2022-12-15

申请号：US17344900

申请日：2021-06-10

Applicant: Apple Inc.

Inventor： Christopher L. Mills

IPC: G06N3/063 ,  G06F13/16

Abstract: Embodiments of the present disclosure relate to circular buffers in a neural processor circuit. The neural processor circuit includes multiple neural engine circuits and a data processor circuit coupled to at least one of the neural engine circuits. The at least one neural engine circuit performs at least convolution operations. The data processor circuit includes a circular buffer, and a flow control circuit coupled to the circular buffer. The flow control circuit generates at least one addressing parameter that defines wrapping of data in the circular buffer. The circular buffer controls data flow in the neural processor circuit by storing first data associated with the at least one neural engine circuit so that the first data is wrapped around in the circular buffer. An addressing layout of the first data wrapped around in the circular buffer is defined by the at least one addressing parameter.

公开(公告)号：US20220237439A1

公开(公告)日：2022-07-28

申请号：US17155896

申请日：2021-01-22

Applicant: Apple Inc.

Inventor： Kenneth W. Waters ,  Christopher L. Mills

IPC: G06N3/063 ,  G06F9/30 ,  G06N3/04 ,  G06F9/48

Abstract: A neural processor includes neural engines for performing convolution operations on input data corresponding to one or more tasks to generate output data. The neural processor circuit also includes a data processor circuit that is coupled to one or more neural engine. The data processor circuit receives the output data from the neural engine and generates a branching command from the output data. The neural processor circuit further includes a task manager that is coupled to the data processor circuit. The task manager receives the branching command from the data processor circuit. The task manager enqueues one of two or more segment branches according to the received branching command. The two or more segment branches are subsequent to a pre-branch task segment that includes the pre-branch task. The task manager transmits a task from the selected one of the segment branches to data processor circuit to perform the task.

公开(公告)号：US20190340491A1

公开(公告)日：2019-11-07

申请号：US15971882

申请日：2018-05-04

Applicant: Apple Inc.

Inventor： Erik K. Norden ,  Liran Fishel ,  Sung Hee Park ,  Jaewon Shin ,  Christopher L. Mills ,  Seungjin Lee ,  Fernando A. Mujica

IPC: G06N3/04 ,  G06N3/08 ,  G06F1/32

Abstract: Embodiments relate to a neural processor circuit with scalable architecture for instantiating one or more neural networks. The neural processor circuit includes a data buffer coupled to a memory external to the neural processor circuit, and a plurality of neural engine circuits. To execute tasks that instantiate the neural networks, each neural engine circuit generates output data using input data and kernel coefficients. A neural processor circuit may include multiple neural engine circuits that are selectively activated or deactivated according to configuration data of the tasks. Furthermore, an electronic device may include multiple neural processor circuits that are selectively activated or deactivated to execute the tasks.

公开(公告)号：US09756266B2

公开(公告)日：2017-09-05

申请号：US14977384

申请日：2015-12-21

Applicant: Apple Inc.

Inventor： Christopher L. Mills ,  Sheng Lin ,  David R. Pope ,  D. Amnon Silverstein ,  Suk Hwan Lim

IPC: H04N5/262 ,  H04N5/345 ,  H04N9/64 ,  H04N5/232 ,  G06T3/40 ,  H04N9/07

CPC classification number: H04N9/04511 ,  G06T3/4015 ,  H04N5/2628 ,  H04N9/07 ,  H04N9/646 ,  H04N2209/046

Abstract: An input rescale module that performs cross-color correlated downscaling of sensor data in the horizontal and vertical dimensions. The module may perform a first-pass demosaic of sensor data, apply horizontal and vertical scalers to resample and downsize the data in the horizontal and vertical dimensions, and then remosaic the data to provide horizontally and vertically downscaled sensor data as output for additional image processing. The module may, for example, act as a front end scaler for an image signal processor (ISP). The demosaic performed by the module may be a relatively simple demosaic, for example a demosaic function that works on 3×3 blocks of pixels. The front end of module may receive and process sensor data at two pixels per clock (ppc); the horizontal filter component reduces the sensor data down to one ppc for downstream components of the input rescale module and for the ISP pipeline.

公开(公告)号：US09386234B2

公开(公告)日：2016-07-05

申请号：US14449022

申请日：2014-07-31

Applicant: Apple Inc.

Inventor： Christopher L. Mills ,  Simon W. Butler

IPC: H04N5/262 ,  H04N5/77 ,  G06T3/00

CPC classification number: H04N5/2628 ,  G06T1/60 ,  G06T3/0056 ,  G06T3/40 ,  G09G5/026 ,  G09G5/397 ,  H04N5/23229 ,  H04N5/772

Abstract: An output rescale module may determine an estimated set of lines to hold in vertical support for use when performing image transformations. For example, an output rescale module may monitor input Y coordinates (in terms of input pixel lines) computed over previous lines and compute a set of lines to hold in a set of line buffers. As each output pixel line is generated, the output rescale module may compute the minimum and maximum values of Y generated by the transform across that line. The minimum and maximum input Y coordinates may then be averaged to determine the center value (the centermost input line) for that output line. The difference (in terms of input pixel lines) between centerlines for two adjacent output lines may be added to the centerline value for the current output line to estimate a center line for the next (not yet generated) output pixel line.

Abstract translation: 输出重定标模块可以确定在执行图像变换时用于垂直支持的线的估计集合用于使用。 例如，输出重定标模块可以监视在先前线路上计算的输入Y坐标（以输入像素线为单位），并计算一组线以保持在一组行缓冲器中。 当产生每个输出像素行时，输出重定标模块可以计算通过该行的变换生成的Y的最小值和最大值。 然后可以对最小和最大输入Y坐标进行平均，以确定该输出线的中心值（最中心输入线）。 两个相邻输出线的中心线之间的差异（以输入像素线为单位）可以被添加到当前输出行的中心线值，以估计下一个（尚未生成的）输出像素行的中心线。

公开(公告)号：US09219870B1

公开(公告)日：2015-12-22

申请号：US14449005

申请日：2014-07-31

Applicant: Apple Inc.

Inventor： Christopher L. Mills ,  Sheng Lin ,  David R. Pope ,  D. Amnon Silverstein ,  Suk Hwan Lim

IPC: H04N5/345 ,  H04N9/64 ,  G06T3/40 ,  H04N9/07

CPC classification number: H04N9/04511 ,  G06T3/4015 ,  H04N5/2628 ,  H04N9/07 ,  H04N9/646 ,  H04N2209/046

Abstract: An input rescale module that performs cross-color correlated downscaling of sensor data in the horizontal and vertical dimensions. The module may perform a first-pass demosaic of sensor data, apply horizontal and vertical scalers to resample and downsize the data in the horizontal and vertical dimensions, and then remosaic the data to provide horizontally and vertically downscaled sensor data as output for additional image processing. The module may, for example, act as a front end scaler for an image signal processor (ISP). The demosaic performed by the module may be a relatively simple demosaic, for example a demosaic function that works on 3×3 blocks of pixels. The front end of module may receive and process sensor data at two pixels per clock (ppc); the horizontal filter component reduces the sensor data down to one ppc for downstream components of the input rescale module and for the ISP pipeline.

Abstract translation: 在水平和垂直维度上执行传感器数据的交叉色相关缩小的输入重定标模块。 模块可以执行传感器数据的第一遍去马赛克，应用水平和垂直缩放器对水平和垂直尺寸的数据进行重新采样和缩小，然后重新绘制数据，以提供水平和垂直缩小的传感器数据作为附加图像处理的输出 。 该模块可以例如用作图像信号处理器（ISP）的前端缩放器。 由模块执行的去镶嵌可以是相对简单的去马赛克，例如在3×3像素块上工作的去马赛克功能。 模块的前端可以以每个时钟两个像素（ppc）接收和处理传感器数据; 水平滤波器组件将传感器数据减少到输入重定标模块的下游组件和ISP管线的一ppc。