Systems, devices, and methods related to a Deep Learning Accelerator and memory are described. An integrated circuit may be configured to execute instructions with matrix operands and configured with: random access memory configured to store instructions executable by the Deep Learning Accelerator and store matrices of an Artificial Neural Network; a connection between the random access memory and the Deep Learning Accelerator; a first interface to a memory controller of a Central Processing Unit; and a second interface to a direct memory access controller. While the Deep Learning Accelerator is using the random access memory to process current input to the Artificial Neural Network in generating current output from the Artificial Neural Network, the direct memory access controller may concurrently load next input into the random access memory; and at the same time, the Central Processing Unit may concurrently retrieve prior output from the random access memory.