Provided is an image processing apparatus and method for filling a hole area in an image. The image processing apparatus may include a candidate pixel setting unit to set a plurality of candidate pixels corresponding to at least one hole pixel in a hole area using neighboring pixels around the hole area, and a pixel value determining unit to determine a pixel value of the hole pixel using the plurality of candidate pixels.