A parallel processing system utilizes a plurality of simultaneously operable arithmetic units to provide matrix-vector products, with each of the arithmetic units implementing the matrix-vector product calculations for plural rows of a matrix stored as vectors in an arithmetic unit. A column of a second matrix is broadcast to the respective arithmetic units whereby the products may be developed in all the arithmetic units simultaneously. The broadcasting of the matrix elements is accomplished via a memory bus which may be employed for selectively or simultaneously accessing registers in the various arithmetic units whereby vector information may be written into memory addresses and calculation results retrieved therefrom.