This disclosure describes techniques for improving mode selection decisions during the encoding of macroblocks (or other blocks) of multimedia frames of a multimedia sequence. During motion estimation, the encoding modes for macroblocks can be determined so that a desirable encoding rate and acceptable levels of distortion (i.e., acceptable rate-distortion) can be achieved. The techniques may include selecting a set of multimedia coding modes between at least two sets of possible multimedia coding modes for a macroblock of a multimedia frame based on a detail metric associated with the macroblock and mode information associated with neighboring blocks to the macroblock.