公开(公告)号：US11651192B2

公开(公告)日：2023-05-16

申请号：US16788261

申请日：2020-02-11

Applicant: Apple Inc.

Inventor： James C. Gabriel ,  Mohammad Rastegari ,  Hessam Bagherinezhad ,  Saman Naderiparizi ,  Anish Prabhu ,  Sophie Lebrecht ,  Jonathan Gelsey ,  Sayyed Karen Khatamifard ,  Andrew L. Chronister ,  David Bakin ,  Andrew Z. Luo

IPC: G06N3/04 ,  G06N3/08

CPC classification number: G06N3/04 ,  G06N3/08

Abstract: Systems and processes for training and compressing a convolutional neural network model include the use of quantization and layer fusion. Quantized training data is passed through a convolutional layer of a neural network model to generate convolutional results during a first iteration of training the neural network model. The convolutional results are passed through a batch normalization layer of the neural network model to update normalization parameters of the batch normalization layer. The convolutional layer is fused with the batch normalization layer to generate a first fused layer and the fused parameters of the fused layer are quantized. The quantized training data is passed through the fused layer using the quantized fused parameters to generate output data, which may be quantized for a subsequent layer in the training iteration.