公开(公告)号：US11461622B2

公开(公告)日：2022-10-04

申请号：US16457268

申请日：2019-06-28

Applicant: Amazon Technologies, Inc.

Inventor： Samuel Jacob ,  Ilya Minkin ,  Mohammad El-Shabani

IPC: G06F1/00 ,  G06N3/063 ,  G06N5/04

Abstract: Embodiments include techniques for enabling execution of N inferences on an execution engine of a neural network device. Instruction code for a single inference is stored in a memory that is accessible by a DMA engine, the instruction code forming a regular code block. A NOP code block and a reset code block for resetting an instruction DMA queue are stored in the memory. The instruction DMA queue is generated such that, when it is executed by the DMA engine, it causes the DMA engine to copy, for each of N inferences, both the regular code block and an additional code block to an instruction buffer. The additional code block is the NOP code block for the first N−1 inferences and is the reset code block for the Nth inference. When the reset code block is executed by the execution engine, the instruction DMA queue is reset.

公开(公告)号：US10664282B1

公开(公告)日：2020-05-26

申请号：US16266731

申请日：2019-02-04

Applicant: Amazon Technologies, Inc.

Inventor： Ilya Minkin ,  Ron Diamant ,  Mohammad El-Shabani ,  Dana Michelle Vantrease

IPC: G06F8/60 ,  G06F9/38 ,  G06F9/30 ,  G06F9/52 ,  G06F13/28 ,  G06F9/50

Abstract: Methods for repeated execution of program code by an execution engine are provided. In order to execute large programs, the instruction buffer of an execution engine may be refilled may times with program code to complete one execution of the program. At completion of program execution, the program code needed to begin re-execution of the program is no longer in the instruction buffer. A runtime driver program can load instructions into the instruction buffer, or can cause instructions to be loaded. Once the instructions are loaded, the execution engine may be able to re-execute the instructions without needing further assistance from the runtime driver.

公开(公告)号：US10592250B1

公开(公告)日：2020-03-17

申请号：US16014646

申请日：2018-06-21

Applicant: Amazon Technologies, Inc.

Inventor： Ron Diamant ,  Ilya Minkin

IPC: G06F9/38 ,  G06F12/1081 ,  G06N5/04 ,  G06F5/06 ,  G06N3/04 ,  G06F13/28

Abstract: Disclosed herein are techniques for self-refilling an instruction buffer by an execution engine while the execution engine executes instructions in the instruction buffer. An instruction loader splits instruction code into sections of code and creates a data store (e.g., a DMA ring) for loading the sections of code into the instruction buffer. In some embodiments, an instruction is added to some sections of code. The instruction, when executed by the execution engine, triggers the loading of one or more sections of code into the instruction buffer based on one or more entries in the data store. In some embodiments, a hardware logic in the execution engine is configured to trigger the loading of the sections of code into the instruction buffer. In some embodiments, the one or more sections of code are loaded into the instruction buffer through a refill page that is different from the instruction buffer.