An exhaust purification system of an internal combustion engine which has a plurality of cylinders is comprised of an exhaust purification catalyst, a downstream side air-fuel ratio sensor, and a control device which controls the average air-fuel ratio of the exhaust gas and the combustion air-fuel ratios of the cylinders. The control device performs average air-fuel ratio control where it alternately controls the average air-fuel ratio between the rich air-fuel ratio and the lean air-fuel ratio and inter-cylinder air-fuel ratio control where it controls the combustion air-fuel ratios of the cylinders so that the combustion air-fuel ratio becomes the rich air-fuel ratio at least at one cylinder among the plurality of cylinders. In average air-fuel ratio control, the average air-fuel ratio is controlled so that the lean shift amount becomes smaller than the rich shift amount when the temperature of the exhaust purification catalyst is low and the average air-fuel ratio is controlled so that the lean shift amount becomes larger than the rich shift amount when the temperature is high.