An apparatus and method are provided for controlling rounding when performing a floating point operation. The apparatus has argument reduction circuitry to perform an argument reduction operation, and in addition provides reduce and round circuitry that generates from a supplied floating point value a modified floating point value to be input to the argument reduction circuitry. The reduce and round circuitry is arranged to modify a significand of the supplied floating point value, based on a specified value N, in order to produce a truncated significand with a specified rounding applied, the truncated significand being N bits shorter than the significand of the supplied floating point value, and then being used as a significand for the modified floating point value. The specified value N is chosen such that the argument reduction operation performed using the modified floating point value will inhibit roundoff error in a result of the argument reduction operation. By enabling roundoff error to be inhibited in such a way, it is possible to use such argument reduction circuitry in the computation of a number of floating point operations while enabling the correct rounded result to be obtained.