Video decoding innovations for using local picture identifiers and computing co-located information are described. In one aspect, a decoder identifies reference pictures in a reference picture list of a temporal direct prediction mode macroblock that match reference pictures used by a co-located macroblock using local picture identifiers. In another aspect, a decoder determines whether reference pictures used by blocks are the same by comparing local picture identifiers during calculation of boundary strength. In yet another aspect, a decoder determines a picture type of a picture and based on the picture type selectively skips or simplifies computation of co-located information for use in reconstructing direct prediction mode macroblocks outside the picture.