Numerous embodiments are disclosed for an analog neuromorphic memory system for use in a deep learning neural network. In one embodiment, the analog neuromorphic memory system comprises a plurality of vector-by-matrix multiplication systems, each vector-by-matrix multiplication system comprising an array of memory cells, a low voltage row decoder, a high voltage row decoder, and a low voltage column decoder; a plurality of output blocks, each output block providing an output in response to at least one of the plurality of vector-by-matrix multiplication systems; and a shared verify block configured to concurrently perform a verify operation after a program operation on two or more of the plurality of vector-by-matrix systems.