To achieve an encoding system including a highly efficient prediction performed in response to the content of a scene, a significance, and a motion characteristic of a moving picture and the like, a reference memory, a motion compensator receiving a parameter representing the motion of a prediction picture segment for generating a predicted picture by using arbitrary data stored in the memory based upon the parameter, and memory update unit that updates picture data in memory and controls the capacity of the memory, are provided.