An apparatus configured to decode video information includes a memory and a processor in communication with the memory. The memory is configured to store video information associated with a bitstream. The apparatus further includes a processor in communication with the memory, the processor configured to determine that a reference layer is not included in the bitstream. The processor is further configured to receive, from an external source, a decoded base layer picture associated with an access unit, to store the decoded base layer picture in a memory. The processor is further configured to decode pictures associated with the access unit based on the stored decoded base layer picture, and subsequent to decoding the pictures associated with the access unit, empty the decoded base layer picture from the memory.