A decrease in purification performance of a system as a whole is suppressed in cases where catalysts are arranged in an upstream side portion and in a downstream side portion of an exhaust passage in an internal combustion engine. A first catalyst, a sensor and a second catalyst are sequentially arranged in the exhaust passage of the internal combustion engine, wherein a switch is made between lean control and rich control in such a manner that an amount of increase or decrease in a storage amount of oxygen falls within a predetermined range, and in cases where the rich control is carried out after performing a predetermined number of times of rich control, the rich control is continued until an air fuel ratio detected by the sensor becomes equal to or less than a rich determination threshold value which is smaller than a stoichiometric air fuel ratio.