After completion of a prediction process for a single line in an image of size S, the DC components and AC components, held in a lower stage part of a predicted luminance value storage are copied into a line part. This copying is done to use the DC components and AC components that have been copied into line part as reference values for the prediction process of the target macro blocks of the next single line. Performing the prediction process while repeating such copying eliminates the need to secure the area for storing of the DC components and AC components of the entire image size in predicted luminance value storage for the prediction process.