A video codec provides for adaptive vertical macroblock alignment of mixed interlaced and progressive video sequences. With adaptive vertical macroblock alignment, a video codec enforces a macroblock alignment height restriction on per picture basis, rather than requiring that all frames in a sequence adhere to a uniform height restriction. The video codec can then apply less padding to progressive and like type pictures that have smaller macroblock alignment increments, than to interlaced type pictures with larger alignment increments, which can save significant compression overhead.