Parametric Power-Of-2 Clipping Activations for Quantization for Convolutional Neural Networks

Invention Application

US20240394543A1 Parametric Power-Of-2 Clipping Activations for Quantization for Convolutional Neural Networks 有权

Please log in to see more content

Patent Title: Parametric Power-Of-2 Clipping Activations for Quantization for Convolutional Neural Networks
Application No.: US18795565

Application Date: 2024-08-06
Publication No.: US20240394543A1

Publication Date: 2024-11-28
Inventor: Manu Mathew , Kumar Desappan , Soyeb Noormohammed Nagori , Debapriya Maji , Pramod Kumar Swami
Applicant: Texas Instruments Incorporated
Applicant Address: US TX Dallas
Assignee: Texas Instruments Incorporated
Current Assignee: Texas Instruments Incorporated
Current Assignee Address: US TX Dallas
Priority: IN201941051538 20191212
Main IPC: G06N3/084
IPC: G06N3/084 ; G06N3/04 ; G06N3/048

Parametric Power-Of-2 Clipping Activations for Quantization for Convolutional Neural Networks

Abstract:

In an example, a method includes executing, using one or more processors, a power-of-2 parametric activation (PACT2) function to quantize a set of data. The executing of the PACT2 function includes determining a distribution for the set of data; discarding a portion of the data corresponding to a tail of the distribution to form a remaining set of data; estimating a maximum value of the remaining set of data; determining a new maximum value of the remaining set of data using a moving average and at least one historical value of at least one prior remaining set of data; determining a clipping value by expanding the new maximum value to a nearest power of two value; and quantizing the set of data using the clipping value to form a quantized set of data.

Information query

Global Dossier Espacenet