A sequence of enhanced dynamic range (EDR) images and a sequence of standard dynamic range images are encoded using a backwards-compatible SDR high-definition (HD) base layer and one or more enhancement layers. The EDR and SDR video signals may be of the same resolution (e.g., HD) or at different resolutions (e.g., 4K and HD) and are encoded using a dual-view-dual-layer (DVDL) encoder to generate a coded base layer (BL) and a coded enhancement layer (EL). The DVDL encoder includes a reference processing unit (RPU) which is adapted to compute a reference stream based on the coded BL stream. The RPU operations include post-processing, normalization, inverse normalization, and image registration. Decoders for decoding the coded BL and EL streams to generate a backwards compatible 2D SDR stream and additional 2D or 3D SDR or EDR streams, are also described.