Disclosed herein is an apparatus and method for a matrix multiplication operation. The apparatus may include memory for storing first matrix data and second matrix data, an X buffer for storing the first matrix data, a Y buffer for storing the second matrix data, multiple operation units for performing Multiply-and-Accumulate (MAC) operations in parallel on the data input from the X buffer and the Y buffer, and a data loader for storing the first matrix data and the second matrix data read from the memory in the X buffer and the Y buffer, respectively.