公开(公告)号：US20220207376A1

公开(公告)日：2022-06-30

申请号：US17134814

申请日：2020-12-28

Applicant: International Business Machines Corporation

Inventor： Tayfun Gokmen ,  Oguzhan Murat Onen ,  Chai Wah Wu ,  Mark S. Squillante ,  Malte Johannes Rasch ,  Tomasz J. Nowicki ,  Wilfried Haensch ,  Lior Horesh ,  Vasileios Kalantzis ,  Vanessa Lopez-Marrero

IPC: G06N3/08 ,  G06N3/063 ,  G06N3/04 ,  G06F17/16

Abstract: Matrix inversion systems and methods are implemented using an analog resistive processing unit (RPU) array for hardware accelerated computing. A request is received from an application to compute an inverse matrix of a given matrix, and a matrix inversion process is performed in response to the received request. The matrix inversion process includes storing a first estimated inverse matrix of the given matrix in an array RPU cells, performing a first iterative process on the first estimated inverse matrix stored in the array of RPU cells to converge the first estimated inverse matrix to a second estimated inverse matrix of the given matrix, and reading the second estimated inverse matrix from the array of RPU cells upon completion of the first iterative process. An inverse matrix is returned to the application, wherein the returned inverse matrix is based, at least in part, on the second estimated inverse matrix.

公开(公告)号：US20220207344A1

公开(公告)日：2022-06-30

申请号：US17134377

申请日：2020-12-26

Applicant: International Business Machines Corporation

Inventor： Tayfun Gokmen

IPC: G06N3/08 ,  G06N3/04 ,  G06N3/063 ,  G06F17/16

Abstract: In one aspect, a method of training a DNN includes transmitting an input vector x through a weight matrix W and reading a resulting output vector y, transmitting an error signal δ, transmitting the input vector x with the error signal δ through conductive row wires of a matrix A, and transmitting an input vector ei and reading a resulting output vector y′ as current output. The training also includes updating a hidden matrix H comprising an H value for RPU devices by iteratively adding the output vector y′ multiplied by the transpose of the input vector ei to each H value. The training also includes, when an H value reaches a threshold value, transmitting the input vector ei as a voltage pulse through the conductive column wires of the matrix W simultaneously with sign information of the H values that reached a threshold value as voltage pulses through the conductive row wires matrix W.

公开(公告)号：US20210357540A1

公开(公告)日：2021-11-18

申请号：US16874819

申请日：2020-05-15

Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION ,  Ramot at Tel-Aviv University Ltd.

Inventor： Lior Horesh ,  Oguzhan Murat Onen ,  Haim Avron ,  Tayfun Gokmen ,  Vasileios Kalantzis ,  Shashanka Ubaru

IPC: G06F30/10

Abstract: A computer-implemented method is presented for performing matrix sketching by employing an analog crossbar architecture. The method includes low rank updating a first matrix for a first period of time, copying the first matrix into a dynamic correction computing device, switching to a second matrix to low rank update the second matrix for a second period of time, as the second matrix is low rank updated, feeding the first matrix with first stochastic pulses to reset the first matrix back to a first matrix symmetry point, copying the second matrix into the dynamic correction computing device, switching back to the first matrix to low rank update the first matrix for a third period of time, and as the first matrix is low rank updated, feeding the second matrix with second stochastic pulses to reset the second matrix back to a second matrix symmetry point.

公开(公告)号：US11062208B2

公开(公告)日：2021-07-13

申请号：US15842724

申请日：2017-12-14

Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION

Inventor： Tayfun Gokmen ,  Oguzhan Murat Önen

IPC: G06N3/08 ,  G06F7/544 ,  G06N3/063 ,  G06N3/04 ,  G06F5/01 ,  G06F17/16

Abstract: A computer-implemented method and computer processing system are provided for update management for a neural network. The method includes performing an isotropic update process on the neural network using a Resistive Processing Unit. The isotropic update process uses a multiplicand and a multiplier from a multiplication operation. The performing step includes scaling the multiplicand and the multiplier to have a same order of magnitude.

公开(公告)号：US10956815B2

公开(公告)日：2021-03-23

申请号：US15609691

申请日：2017-05-31

Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION

Inventor： Tayfun Gokmen

IPC: G06N3/08 ,  G06N3/063

Abstract: Technical solutions are described for improving efficiency of training a resistive processing unit (RPU) array using a neural network training methodology. An example method includes reducing asymmetric RPUs from the RPU array by determining an asymmetric value of an RPU from the RPU array, and burning the RPU in response to the asymmetry value being above a predetermined threshold. The RPU can be burned by causing an electric voltage across the RPU to be above a predetermined limit. The method further includes initiating the training methodology for the RPU array after the asymmetric RPUs from the RPU array are reduced.

公开(公告)号：US10901939B2

公开(公告)日：2021-01-26

申请号：US14928970

申请日：2015-10-30

Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION

Inventor： Tayfun Gokmen

IPC: G06F15/80 ,  G06F15/78 ,  G06F9/38 ,  G11C13/00 ,  G06N3/04

Abstract: A processor includes an array of resistive processing units connected between row and column lines with a resistive element. A first single instruction, multiple data processing unit (SIMD) is connected to the row lines. A second SIMD is connected to the column lines. A first instruction issuer is connected to the first SIMD to issue instructions to the first SIMD, and a second instruction issuer is connected to the second SIMD to issue instructions to the second SIMD such that the processor is programmable and configurable for specific operations depending on an issued instruction set.

公开(公告)号：US20190325291A1

公开(公告)日：2019-10-24

申请号：US15958322

申请日：2018-04-20

Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION

Inventor： Tayfun Gokmen ,  Wilfried Haensch ,  Seyoung Kim

IPC: G06N3/04 ,  G06N3/08

Abstract: Embodiments of the present invention include a crossbar array that includes a resistive processing unit (RPU) device at each crosspoint in the crossbar array. The RPU device includes a single weight storage element, and multiple weight reader elements. A first weight reader element is coupled with a first row wire to compute a first matrix product value using a first value and a stored value, the first value being transmitted via the first row wire and the stored value being stored in the single weight storage element. A second weight reader element is coupled with a second row wire to compute a second matrix product value of a second value and said stored value, the second value being transmitted via the second row wire.

公开(公告)号：US20190318239A1

公开(公告)日：2019-10-17

申请号：US15954170

申请日：2018-04-16

Applicant: International Business Machines Corporation

Inventor： Seyoung Kim ,  Tayfun Gokmen

IPC: G06N3/08 ,  G06N3/063 ,  G06N3/04

Abstract: Systems and methods are provided to perform weight update operations in a resistive processing unit (RPU) system to update weight values of RPU devices comprising tunable resistive device. A weight update operation for a given RPU device includes maintaining a weight update accumulation value for the RPU device, adjusting the weight update accumulation value by one unit update value in response to a detected coincidence of stochastic bits streams of input vectors applied on an update row and update column control lines connected to the RPU device, generating a weight update control signal in response to the accumulated weight value reaching a predefined threshold value, and adjusting a conductance level of the tunable resistive device by one unit conductance value in response to the weight update control signal, wherein the one unit conductance value corresponds to one unit weight value of the RPU device.

公开(公告)号：US10325007B2

公开(公告)日：2019-06-18

申请号：US15479561

申请日：2017-04-05

Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION

Inventor： Tayfun Gokmen ,  Oguzhan Murat Onen

IPC: G06G7/16 ,  G06N3/08 ,  G06F17/16 ,  G06F7/523 ,  G06F7/535 ,  G06F7/544 ,  G06N3/063

Abstract: A method, computer program product, and circuit are provided for noise and bound management for a Resistive Processing Unit (RN) array having an op-amp. The method includes reducing the noise in an output signal from the RPU array by using a largest value, in a sigma vector having a plurality of values, as a representation for a window for an input signal to the RPU array. The input signal to the RPU array is formed from the plurality of values. The method further includes sensing saturation at an output of the op-amp. The method also includes managing the bound to eliminate the saturation by reducing the plurality of values from which the input sign to the RPU is formed.

公开(公告)号：US20180293208A1

公开(公告)日：2018-10-11

申请号：US15479561

申请日：2017-04-05

Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION

Inventor： Tayfun Gokmen ,  Oguzhan Murat Onen

IPC: G06F17/16 ,  G06F7/535 ,  G06F7/523 ,  G06N3/08

CPC classification number: G06F17/16 ,  G06F7/523 ,  G06F7/535 ,  G06F7/5443 ,  G06F2207/4814 ,  G06F2207/4824 ,  G06F2207/4828 ,  G06G7/16 ,  G06N3/0635 ,  G06N3/08

Abstract: A method, computer program product, and circuit are provided for noise and bound management for a Resistive Processing Unit (RN) array having an op-amp. The method includes reducing the noise in an output signal from the RPU array by using a largest value, in a sigma vector having a plurality of values, as a representation for a window for an input signal to the RPU array. The input signal to the RPU array is formed from the plurality of values. The method further includes sensing saturation at an output of the op-amp. The method also includes managing the bound to eliminate the saturation by reducing the plurality of values from which the input sign to the RPU is formed.