Scalable video coding based on H.264/AVC uses motion compensation, which results in motion vectors and residual pictures. To optimize the encoding of the residual pictures of the enhancement-layer, they can be predicted from the base-layer. This requires complex encoders and decoders. Simplified residual prediction techniques are provided that focus on reducing the encoding and decoding complexity of the spatial scalable enhancement-layer with optimized coding efficiency. The method for encoding video data containing high-pass frames and low-pass frames comprises the steps of encoding the low-pass frames, wherein residual prediction may be used, splitting the high-pass frames into two interleaving frame groups, encoding the frames of the first of said frame groups, wherein residual prediction may be used, and encoding the frames of the second of the frame groups using an encoding method without residual prediction.