The present invention is to provide an image processing apparatus using the difference among scaled images as a layered image and a method thereof, which utilize the Gaussian and Laplacian pyramid theory to convert an original image into a plurality of scaled images of different scales, and the difference among scaled images of two adjacent different scales as a layered image of the corresponding layer, so that the edge and line characteristics of a scene of the original image for each layered image can be displayed in different levels sequentially from a clear level to a vague level, and provide a layered image display interface and an image characteristic editing interface for users to examine each layered image through the layered image display interface and edit or perform special effect to each layered image, so as to simulate different visual effects based on different vision models.