A reference picture marking process and a reference picture list management process is handled in a unified reference picture marking and reference picture list management process. A new idle reference picture list may be used for handling reference pictures that are not used for reference in the current picture. Differential coding of picture order count may be used to increase coding efficiency. The reference picture management syntax structure may be sent in the picture parameter set for improved coding efficiency e.g. in regular GOP (group of pictures) arrangements.