A neural network operation apparatus and method are disclosed. A neural network operation apparatus includes a receiver that receives data for a neural network operation, and a processor that performs a scaling operation by multiplying the data by a constant, performs a rounding operation by truncating bits forming a result of the scaling operation, performs a scaling back operation based on a result of the rounding operation, and generates a neural network operation result by accumulating results of the scaling back operation.