Techniques and systems are provided for performing predictive random access using a background picture. For example, a method of decoding video data includes obtaining an encoded video bitstream comprising a plurality of pictures. The plurality of pictures include a plurality of predictive random access pictures. A predictive random access picture is at least partially encoded using inter-prediction based on at least one background picture. The method further includes determining, for a time instance of the video bitstream, a predictive random access picture of the plurality of predictive random access pictures with a time stamp closest in time to the time instance. The method further includes determining a background picture associated with the predictive random access picture, and decoding at least a portion of the predictive random access picture using inter-prediction based on the background picture.