Embodiments detailed herein relate to matrix operations. For example, an apparatus comprises programmable configuration storage, decode circuitry and execution circuitry. The programmable configuration storage is to store configuration information for a first matrix, a second matrix, and a third matrix, the configuration information including a first value corresponding to a first number of rows for the first matrix, a second value corresponding to a second number of columns for the first matrix, a third value corresponding to a third number of rows for the second matrix, a fourth value corresponding to a fourth number of columns for the second matrix, a fifth value corresponding to a fifth number of rows for the third matrix, a sixth value corresponding to the sixth number of columns for the third matrix, and a start row value corresponding to a row of a corresponding matrix at which to restart execution of at least one of a plurality of matrix instructions. The decode circuitry is to decode the plurality of matrix instructions, including a single instruction to perform dot-product and accumulation, the single instruction having a first operand to specify a first register, a second operand to specify a second register, and a third operand to specify a third register. The execution circuitry is to perform one or more operations corresponding to the single instruction, including: performing dot-products on elements of the second matrix from the second register and elements of the third matrix from the third register to generate one or more resulting elements, and accumulating the one or more resulting elements into the first matrix in the first register.