Techniques are described related to deriving a reference picture set. A reference picture set may identify reference pictures that can potentially be used to inter-predict a current picture and picture following the current picture in decoding order. In some examples, deriving the reference picture set may include constructing a plurality of reference picture subsets that together form the reference picture set.