A coding distortion removing method is provided for removing coding distortion in two adjacent blocks located on both sides of a block boundary between the two adjacent blocks in a reconstructed image. The method includes deriving a difference of pixel values between a pixel in a first block of the reconstructed image and a pixel in a second block of the reconstructed image adjacent to the first block, deriving an average value of the first quantization parameter and the second quantization parameter, and setting a threshold value to the average value of the first quantization parameter and the second quantization parameter. The method also includes comparing the difference of pixel values with the threshold value, and removing a coding distortion in an area disposed on both sides of the block boundary between the first block and the second block, by applying a filter for coding distortion removal.