Provided are an apparatus and method for enhancing image quality of an input image captured by an image capturing device having an aperture with a plurality of openings employing different color filters. The image quality enhancing apparatus includes an ROI (region of interest) classifying unit classifying ROIs according to distances from the image capturing device of a captured input image by using a cluster-based image segmentation algorithm, a color channel registering unit determining motion vectors of color channels corresponding to the respective ROIs by analyzing a high frequency component of image data included in the ROIs, and moving the color channels based on the determined motion vectors to obtain a registered image of the respective ROIs, and an image fusing unit fusing the registered images of the respective ROIs to obtain a fused image.