A method of determining an offset includes dividing a current image into a plurality of blocks, determining a category of pixels in each of the plurality of blocks based on values of neighboring pixels, determining an offset value for pixels belonging to the category, and adjusting the offset value based on characteristics of the category and a background pixel value of each of the pixels. The offset value is an average of differences between values of original images and restored images of pixels belonging to one category. The background pixel value is an average of values of pixels in a background pixel block to which the pixels belong among background pixel blocks divided to calculate a background pixel value.