A device and method of removing chromatic aberration which occurs in an image generated by capturing a target subject. The chromatic aberration removing device includes a chromatic aberration region detecting unit which detects a chromatic aberration occurrence region in an input image, a coefficient calculating unit which calculates a minimizing pixel value which minimizes a difference between sizes, numbers of pixels or intensities of edges of red, green, and blue (RGB) channels of the chromatic aberration occurrence region, a chroma signal processing unit which removes chromatic aberration in the chromatic aberration occurrence region, and a purple fringing removing unit which removes purple fringing.