A video encoder and corresponding method are provided for encoding an input picture or image block using a prediction from a reference-only picture, where the encoder includes a picture buffer for storing a previously coded picture, and a reference processing unit in signal communication with the picture buffer for generating the reference-only picture from a previously coded picture; and the corresponding method includes receiving a substantially uncompressed image block, filtering (714) a previously coded picture to create an adaptive reference, motion compensating the adaptive reference, subtracting the motion compensated adaptive reference from the substantially uncompressed image block, and encoding the difference between the substantially uncompressed image block and the motion compensated adaptive reference. Video decoder and corresponding method are also disclosed.