-
公开(公告)号：US20170061567A1
公开(公告)日：2017-03-02
申请号：US14836915
申请日：2015-08-26
Applicant: APPLE INC.
Inventor： Suk Hwan Lim , Christopher L. Mills , D. Amnon Silverstein , David R. Pope , Sheng Lin
CPC classification number: G06T1/20 , G06T1/60 , G06T3/4015 , H04N5/23229 , H04N5/3765 , H04N5/917 , H04N9/045
Abstract: An image processing pipeline may process image data at multiple rates. A stream of raw pixel data collected from an image sensor for an image frame may be processed through one or more pipeline stages of an image signal processor. The stream of raw pixel data may then be converted into a full-color domain and scaled to a data size that is less than an initial data size for the image frame. The converted pixel data may be processed through one or more other pipelines stages and output for storage, further processing, or display. In some embodiments, a back-end interface may be implemented as part of the image signal processor via which image data collected from sources other than the image sensor may be received and processed through various pipeline stages at the image signal processor.
Abstract translation: 图像处理流水线可以以多个速率处理图像数据。 从图像帧的图像传感器收集的原始像素数据流可以通过图像信号处理器的一个或多个流水线级进行处理。 然后可以将原始像素数据流转换为全色域并缩放为小于图像帧的初始数据大小的数据大小。 转换的像素数据可以通过一个或多个其他管道级进行处理，并输出用于存储，进一步处理或显示。 在一些实施例中，后端接口可以被实现为图像信号处理器的一部分，通过该图像信号处理器从除图像传感器之外的源收集的图像数据可以通过图像信号处理器处的各种流水线级被接收和处理。
-
公开(公告)号：US09462189B2
公开(公告)日：2016-10-04
申请号：US14449024
申请日：2014-07-31
Applicant: Apple Inc.
Inventor： Christopher L. Mills , David R. Pope , D. Amnon Silverstein
CPC classification number: H04N5/2329 , G06T3/00 , H04N5/23254 , H04N5/23258 , H04N5/23267
Abstract: An image signal processor of a device, apparatus, or computing system that includes a camera capable of capturing image data may apply piecewise perspective transformations to image data received from the camera's image sensor. A scaling unit of an Image Signal Processor (ISP) may perform piecewise perspective transformations on a captured image to correct for rolling shutter artifacts and to provide video image stabilization. Image data may be divided into a series of horizontal slices and perspective transformations may be applied to each slice. The transformations may be based on motion data determined in any of various manners, such as by using gyroscopic data and/or optical-flow calculations. The piecewise perspective transforms may be encoded as Digital Difference Analyzer (DDA) steppers and may be implemented using separable scalar operations. The image signal processor may not write the received image data to system memory until after the transformations have been performed.
Abstract translation: 包括能够拍摄图像数据的相机的设备，装置或计算系统的图像信号处理器可以对从相机的图像传感器接收的图像数据应用分段透视变换。 图像信号处理器（ISP）的缩放单元可以对捕获的图像执行分段透视变换，以校正滚动快门伪影并提供视频图像稳定。 图像数据可以被划分为一系列水平切片，并且透视变换可以应用于每个切片。 变换可以基于以各种方式中的任何一种确定的运动数据，例如通过使用陀螺仪数据和/或光流计算。 分段透视变换可以被编码为数字差分分析器（DDA）步进器，并且可以使用可分离的标量运算来实现。 图像信号处理器可能不会将接收的图像数据写入系统存储器直到转换完成。
-
公开(公告)号：US20160037085A1
公开(公告)日：2016-02-04
申请号：US14449022
申请日：2014-07-31
Applicant: Apple Inc.
Inventor： Christopher L. Mills , Simon W. Butler
CPC classification number: H04N5/2628 , G06T1/60 , G06T3/0056 , G06T3/40 , G09G5/026 , G09G5/397 , H04N5/23229 , H04N5/772
Abstract: An output rescale module may determine an estimated set of lines to hold in vertical support for use when performing image transformations. For example, an output rescale module may monitor input Y coordinates (in terms of input pixel lines) computed over previous lines and compute a set of lines to hold in a set of line buffers. As each output pixel line is generated, the output rescale module may compute the minimum and maximum values of Y generated by the transform across that line. The minimum and maximum input Y coordinates may then be averaged to determine the center value (the centermost input line) for that output line. The difference (in terms of input pixel lines) between centerlines for two adjacent output lines may be added to the centerline value for the current output line to estimate a center line for the next (net yet generated) output pixel line.
Abstract translation: 输出重定标模块可以确定在执行图像变换时用于垂直支持的线的估计集合用于使用。 例如，输出重定标模块可以监视在先前线路上计算的输入Y坐标（以输入像素线为单位），并计算一组线以保持在一组行缓冲器中。 当产生每个输出像素行时，输出重定标模块可以计算通过该行的变换生成的Y的最小值和最大值。 然后可以对最小和最大输入Y坐标进行平均，以确定该输出线的中心值（最中心输入线）。 可以将两个相邻输出线的中心线之间的差异（以输入像素线为单位）添加到当前输出行的中心线值，以估计下一（已生成）输出像素行的中心线。
-
公开(公告)号：US20160037073A1
公开(公告)日：2016-02-04
申请号：US14449024
申请日：2014-07-31
Applicant: Apple Inc.
Inventor： Christopher L. Mills , David R. Pope , D. Amnon Silverstein
IPC: H04N5/232
CPC classification number: H04N5/2329 , G06T3/00 , H04N5/23254 , H04N5/23258 , H04N5/23267
Abstract: An image signal processor of a device, apparatus, or computing system that includes a camera capable of capturing image data may apply piecewise perspective transformations to image data received from the camera's image sensor. A scaling unit of an Image Signal Processor (ISP) may perform piecewise perspective transformations on a captured image to correct for rolling shutter artifacts and to provide video image stabilization. Image data may be divided into a series of horizontal slices and perspective transformations may be applied to each slice. The transformations may be based on motion data determined in any of various manners, such as by using gyroscopic data and/or optical-flow calculations. The piecewise perspective transforms may be encoded as Digital Difference Analyzer (DDA) steppers and may be implemented using separable scalar operations. The image signal processor may not write the received image data to system memory until after the transformations have been performed.
Abstract translation: 包括能够拍摄图像数据的相机的设备，装置或计算系统的图像信号处理器可以对从相机的图像传感器接收的图像数据应用分段透视变换。 图像信号处理器（ISP）的缩放单元可以对捕获的图像执行分段透视变换，以校正滚动快门伪影并提供视频图像稳定。 图像数据可以被划分为一系列水平切片，并且透视变换可以应用于每个切片。 变换可以基于以各种方式中的任何一种确定的运动数据，例如通过使用陀螺仪数据和/或光流计算。 分段透视变换可以被编码为数字差分分析器（DDA）步进器，并且可以使用可分离的标量运算来实现。 图像信号处理器可能不会将接收的图像数据写入系统存储器直到转换完成。
-
公开(公告)号：US12277494B2
公开(公告)日：2025-04-15
申请号：US16953033
申请日：2020-11-19
Applicant: Apple Inc.
Inventor： Christopher L. Mills
Abstract: Embodiments of the present disclosure relate to a tensor access operation circuit in a neural processor circuit. The neural processor circuit further includes a data processor circuit and at least one neural engine circuit. The tensor access operation circuit indirectly accesses at least a region of a source tensor in a system memory having a rank, and maps one or more source components of the source tensor into an input tensor having another rank. The data processor circuit stores an output version of the input tensor obtained from the tensor access operation circuit and sends the output version of the input tensor as multiple of units of input data to the at least one neural engine circuit. The at least one neural engine circuit performs at least convolution operations on the units of input data and at least one kernel to generate output data.
-
Patent Agency Ranking
公开(公告)号：US12210959B2
公开(公告)日：2025-01-28
申请号：US17155896
申请日：2021-01-22
Applicant: Apple Inc.
Inventor： Kenneth W. Waters , Christopher L. Mills
Abstract: A neural processor includes neural engines for performing convolution operations on input data corresponding to one or more tasks to generate output data. The neural processor circuit also includes a data processor circuit that is coupled to one or more neural engine. The data processor circuit receives the output data from the neural engine and generates a branching command from the output data. The neural processor circuit further includes a task manager that is coupled to the data processor circuit. The task manager receives the branching command from the data processor circuit. The task manager enqueues one of two or more segment branches according to the received branching command. The two or more segment branches are subsequent to a pre-branch task segment that includes the pre-branch task. The task manager transmits a task from the selected one of the segment branches to data processor circuit to perform the task.
-
公开(公告)号：US11989640B2
公开(公告)日：2024-05-21
申请号：US17991373
申请日：2022-11-21
Applicant: Apple Inc.
Inventor： Erik Norden , Liran Fishel , Sung Hee Park , Jaewon Shin , Christopher L. Mills , Seungjin Lee , Fernando A. Mujica
IPC: G06N3/04 , G06F1/3296 , G06N3/08
CPC classification number: G06N3/04 , G06F1/3296 , G06N3/08
Abstract: Embodiments relate to a neural processor circuit with scalable architecture for instantiating one or more neural networks. The neural processor circuit includes a data buffer coupled to a memory external to the neural processor circuit, and a plurality of neural engine circuits. To execute tasks that instantiate the neural networks, each neural engine circuit generates output data using input data and kernel coefficients. A neural processor circuit may include multiple neural engine circuits that are selectively activated or deactivated according to configuration data of the tasks. Furthermore, an electronic device may include multiple neural processor circuits that are selectively activated or deactivated to execute the tasks.
-
公开(公告)号：US11853868B2
公开(公告)日：2023-12-26
申请号：US17944889
申请日：2022-09-14
Applicant: Apple Inc.
Inventor： Christopher L. Mills , Sung Hee Park
CPC classification number: G06N3/063 , G06F7/5443 , G06F17/15
Abstract: Embodiments of the present disclosure relate to a neural engine of a neural processor circuit having multiple multiply-add circuits and an accumulator circuit coupled to the multiply-add circuits. The multiply-add circuits perform multiply-add operations of a three dimensional convolution on a work unit of input data using a kernel to generate at least a portion of output data in a processing cycle. The accumulator circuit includes multiple batches of accumulators. Each batch of accumulators receives and stores, after the processing cycle, the portion of the output data for each output depth plane of multiple output depth planes. A corresponding batch of accumulators stores, after the processing cycle, the portion of the output data for a subset of the output channels and for each output depth plane.
-
公开(公告)号：US11783174B2
公开(公告)日：2023-10-10
申请号：US15971332
申请日：2018-05-04
Applicant: Apple Inc.
Inventor： Christopher L. Mills
Abstract: Embodiments of the present disclosure relate to splitting input data into smaller units for loading into a data buffer and neural engines in a neural processor circuit for performing neural network operations. The input data of a large size is split into slices and each slice is again split into tiles. The tile is uploaded from an external source to a data buffer inside the neural processor circuit but outside the neural engines. Each tile is again split into work units sized for storing in an input buffer circuit inside each neural engine. The input data stored in the data buffer and the input buffer circuit is reused by the neural engines to reduce re-fetching of input data. Operations of splitting the input data are performed at various components of the neural processor circuit under the management of rasterizers provided in these components.
-
公开(公告)号：US20230128047A1
公开(公告)日：2023-04-27
申请号：US17509598
申请日：2021-10-25
Applicant: Apple Inc.
Inventor： Christopher L. Mills
IPC: G06N3/063
Abstract: Embodiments of the present disclosure relate to binary comparison operations (e.g., Boolean operations) and reduction operations in a neural processor circuit to enable implementation of conditional operations without software control. The neural processor circuit includes a neural engine circuit and a planar engine circuit coupled to the neural engine circuit. The neural engine circuit performs a convolution operation to generate output data. The planar engine circuit includes a binary comparator circuit and a filter circuit coupled to the binary comparator circuit. The binary comparator circuit performs a binary comparison operation on a tensor from the output data to generate a conditional tensor. The filter circuit performs a reduction operation for each patch of the conditional tensor to generate a respective reduced value of multiple reduced values associated with a corresponding channel of multiple channels of the conditional tensor.
-
-
-
-
-
-
-
-
-
Search Results
69
records
(took 0.014 seconds)
