A moving image encoding and decoding system reduces compressed moving picture data amounts by producing a prediction signal suitable for moving pictures including dynamic textures. The system includes a moving picture encoding device 1 that includes a block divider 102 receiving a target pixel signal, a prediction signal generator 103 producing a prediction signal, a subtractor 105 producing a differential signal, and a transformer 106 and quantizer 107 producing an encoded differential signal. The encoding device 1 also includes an inverse quantizer 108 and inverse transformer 109 producing a decoded differential signal, an adder 110 producing a reproduction signal, a frame memory 104 storing the reproduction signal as a reference image, and a reference image generator 113. The reference image generator 113 may obtain an observation matrix C by using a pre-existing reference image. The encoding system 1 produces a new reference image y″t+5 based on the observation matrix C and a state vector xt+5, and stores the new reference image in the frame memory 104. The prediction signal generator 103 produces the prediction signal by using at least the new reference image y″t+5.