A matrix computing device and an operation method for the matrix computing device are provided. The matrix computing device includes a storage unit, a control circuit, and a computing circuit. The storage unit includes a weight matrix. The control circuit re-orders an arrangement order of weights in the weight matrix according to a shape of an output matrix to determine a weight readout order of the weights. The computing circuit receives the weights based on the weight readout order, and performs a matrix computation on the weights and an input matrix to generate a computing matrix. The control circuit performs a reshape transformation on the computing matrix to generate the output matrix, and writes the output matrix to the storage unit.