An arithmetic operation device causes a convolution arithmetic unit to perform a convolution arithmetic operation between a filter and target data corresponding to a size of the filter in each of a plurality of convolution layers constituting a neural network. The arithmetic operation device includes: a bit reduction unit that reduces a bit string corresponding to a first bit number from a least significant bit of the target data and reduces a bit string corresponding to a second bit number from a least significant bit of a weight that is an element of the filter for each convolution layer; and a bit addition unit that adds a bit string corresponding to a third bit number obtained by adding the first bit number and the second bit number to a least significant bit of a convolution arithmetic operation result output from the convolution arithmetic unit by inputting the target data and the weight after being reduced by the bit reduction unit to the convolution arithmetic unit.