公开(公告)号：US11120327B2

公开(公告)日：2021-09-14

申请号：US15971657

申请日：2018-05-04

Applicant: Apple Inc.

Inventor： Liran Fishel ,  Sung Hee Park ,  Christopher L. Mills

IPC: G06N3/04

Abstract: Embodiments relate to a neural processor circuit that includes a kernel access circuit and multiple neural engine circuits. The kernel access circuit reads compressed kernel data from memory external to the neural processor circuit. Each neural engine circuit receives compressed kernel data from the kernel access circuit. Each neural engine circuit includes a kernel extract circuit and a kernel multiply-add (MAD) circuit. The kernel extract circuit extracts uncompressed kernel data from the compressed kernel data. The kernel MAD circuit receives the uncompressed kernel data from the kernel extract circuit and performs neural network operations on a portion of input data using the uncompressed kernel data.

公开(公告)号：US20210271958A1

公开(公告)日：2021-09-02

申请号：US16806798

申请日：2020-03-02

Applicant: Apple Inc.

Inventor： Christopher L. Mills ,  Kenneth W. Waters

IPC: G06N3/063 ,  G06N3/08 ,  G06N5/04 ,  G06F9/38 ,  G06F9/48

Abstract: Embodiments relate to a neural processor circuit including one or more planar engine circuits that perform non-convolution operations in parallel with convolution operations performed by one or more neural engine circuits. The neural engine circuits perform the convolution operations on neural input data corresponding to one or more neural engine tasks to generate neural output data. The planar engine circuits perform non-convolution operations on planar input data corresponding to one or more planar engine tasks to generate planar output data. A data processor circuit in the neural processor circuit addresses data dependency between the one or more neural engine tasks and the one or more planar engine tasks by controlling reading of the neural output data as the planar input data by the planar engine circuits or reading of the planar output data as the neural input data by the neural engine circuits.

公开(公告)号：US20210132945A1

公开(公告)日：2021-05-06

申请号：US16673499

申请日：2019-11-04

Applicant: Apple Inc.

Inventor： Christopher L. Mills

IPC: G06F9/30 ,  G06N3/063

Abstract: Embodiments of the present disclosure relate to chained buffers in a neural processor circuit. The neural processor circuit includes multiple neural engines, a planar engine, a buffer memory, and a flow control circuit. At least one neural engine operates as a first producer of first data or a first consumer of second data. The planar engine operates as a second consumer receiving the first data from the first producer or a second producer sending the second data to the first consumer. Data flow between the at least one neural engine and the planar engine is controlled using at least a subset of buffers in the buffer memory operating as at least one chained buffer that chains flow of the first data and the second data between the at least one neural engine and the planar engine.

公开(公告)号：US20210125041A1

公开(公告)日：2021-04-29

申请号：US16662789

申请日：2019-10-24

Applicant: Apple Inc.

Inventor： Christopher L. Mills ,  Sung Hee Park

IPC: G06N3/063 ,  G06F7/544 ,  G06F17/15

Abstract: Embodiments of the present disclosure relate to a neural engine of a neural processor circuit having multiple multiply-add circuits and an accumulator circuit coupled to the multiply-add circuits. The multiply-add circuits perform multiply-add operations of a three dimensional convolution on a work unit of input data using a kernel to generate at least a portion of output data in a processing cycle. The accumulator circuit includes multiple batches of accumulators. Each batch of accumulators receives and stores, after the processing cycle, the portion of the output data for each output depth plane of multiple output depth planes. A corresponding batch of accumulators stores, after the processing cycle, the portion of the output data for a subset of the output channels and for each output depth plane.

公开(公告)号：US20210103803A1

公开(公告)日：2021-04-08

申请号：US16596439

申请日：2019-10-08

Applicant: Apple Inc.

Inventor： Christopher L. Mills ,  Kenneth W. Waters ,  Youchang Kim

IPC: G06N3/063 ,  G06N20/00 ,  G06F17/15

Abstract: Embodiments relate to a neural processor that include a plurality of neural engine circuits and one or more planar engine circuits. The plurality of neural engine circuits can perform convolution operations of input data of the neural engine circuits with one or more kernels to generate outputs. The planar engine circuit is coupled to the plurality of neural engine circuits. The planar engine circuit generates an output from input data that corresponds to output of the neural engine circuits or a version of input data of the neural processor. The planar engine circuit can be configured to multiple modes. In a pooling mode, the planar engine circuit reduces a spatial size of a version of the input data. In an elementwise mode, the planar engine circuit performs an elementwise operation on the input data. In a reduction mode, the planar engine circuit reduces the rank of a tensor.

公开(公告)号：US10540742B2

公开(公告)日：2020-01-21

申请号：US15499459

申请日：2017-04-27

Applicant: Apple Inc.

Inventor： Christopher L. Mills

IPC: G06T3/00

Abstract: A device that includes integrated circuit includes a tiler circuit, a grid generator, and a warper circuit. The tiler circuit divides the distorted input image data into a plurality of image tiles and stores the image tiles into a memory device. Each image tile is an M×N array of pixel samples where M and N are greater than 1. The grid generator produces a mesh grid that describes a mapping of first pixel locations of the distorted image data to second pixel locations of the corrected image data. The warper circuit reads one or more of the image tiles from the memory device based on the mesh grid and interpolates a warped output image from the image tiles read from memory.

公开(公告)号：US20190340488A1

公开(公告)日：2019-11-07

申请号：US15971657

申请日：2018-05-04

Applicant: Apple Inc.

Inventor： Liran Fishel ,  Sung Hee Park ,  Christopher L. Mills

IPC: G06N3/04

Abstract: Embodiments relate to a neural processor circuit that includes a kernel access circuit and multiple neural engine circuits. The kernel access circuit reads compressed kernel data from memory external to the neural processor circuit. Each neural engine circuit receives compressed kernel data from the kernel access circuit. Each neural engine circuit includes a kernel extract circuit and a kernel multiply-add (MAD) circuit. The kernel extract circuit extracts uncompressed kernel data from the compressed kernel data. The kernel MAD circuit receives the uncompressed kernel data from the kernel extract circuit and performs neural network operations on a portion of input data using the uncompressed kernel data.

公开(公告)号：US20180315170A1

公开(公告)日：2018-11-01

申请号：US15499459

申请日：2017-04-27

Applicant: Apple Inc.

Inventor： Christopher L. Mills

IPC: G06T5/00 ,  G06T1/60 ,  G06T3/00

CPC classification number: G06T3/0093

Abstract: A device that includes integrated circuit includes a tiler circuit, a grid generator, and a warper circuit. The tiler circuit divides the distorted input image data into a plurality of image tiles and stores the image tiles into a memory device. Each image tile is an M×N array of pixel samples where M and N are greater than 1. The grid generator produces a mesh grid that describes a mapping of first pixel locations of the distorted image data to second pixel locations of the corrected image data. The warper circuit reads one or more of the image tiles from the memory device based on the mesh grid and interpolates a warped output image from the image tiles read from memory.

公开(公告)号：US20130329098A1

公开(公告)日：2013-12-12

申请号：US13724574

申请日：2012-12-21

Applicant: APPLE INC.

Inventor： Suk Hwan Lim ,  Christopher L. Mills ,  Sheng Lin

IPC: H04N9/64

CPC classification number: H04N9/64 ,  H04N5/142 ,  H04N5/23229 ,  H04N5/23254 ,  H04N5/365 ,  H04N5/367 ,  H04N9/045 ,  H04N9/735

Abstract: The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may detect and correct a defective pixel of image data acquired using an image sensor. The image processing pipeline may receive an input pixel of the image data acquired using the image sensor. The image processing pipeline may then identify a set of neighboring pixels having the same color component as the input pixel and remove two neighboring pixels from the set of neighboring pixels thereby generating a modified set of neighboring pixels. Here, the two neighboring pixels correspond to a maximum pixel value and a minimum pixel value of the set of neighboring pixels. The image processing pipeline may then determine a gradient for each neighboring pixel in the modified set of neighboring pixels and determine whether the input pixel includes a dynamic defect or a speckle based at least in part on the gradient for each neighboring pixel in the modified set of neighboring pixels.

Abstract translation: 本公开通常涉及用于图像数据处理的系统和方法。 在某些实施例中，图像处理流水线可以检测和校正使用图像传感器获取的图像数据的缺陷像素。 图像处理流水线可以接收使用图像传感器获取的图像数据的输入像素。 然后，图像处理流水线可以识别具有与输入像素相同颜色分​​量的一组相邻像素，并从相邻像素组中移除两个相邻像素，从而生成修改后的相邻像素集合。 这里，两个相邻像素对应于该组相邻像素的最大像素值和最小像素值。 然后，图像处理流水线可以确定相邻像素的修改集合中的每个相邻像素的梯度，并且确定输入像素是否至少部分地基于修改的集合中的每个相邻像素的梯度的动态缺陷或散斑 相邻像素。