Provided are an apparatus and method for processing an image. The apparatus includes a detector configured to form one group with pieces of pixel information on an edge of a received image, detect line segments, and generate a plurality of curves using the detected line segments, an optimizer configured to apply a weight to a cost function based on the plurality of generated curves according to distance from the center of the image, define a curve having a shorter length than a threshold value as an outlier and a curve having a greater length than the threshold value as an inlier, remove the outlier, and estimate a distortion level parameter, and a corrector configured to correct a position of a pixel with a correction value using the distortion level parameter. Accordingly, it is possible to correct lens distortion of even an image that has not been directly photographed, and to correct images generated by devices equipped with any lens rather than a specific camera or photographing apparatus.