A method and apparatus wherein the method includes the steps of parsing a stream of compressed video, obtaining macroblock size information from the parsed stream, computing factors derived from the macroblock size, wherein the factors include a normalized bit size, a bit size ratio and a neighbor score, computing corresponding adaptive threshold values derived from the relative frame characteristics of the compressed video, comparing the factors derived from the macroblock size information with the corresponding adaptive threshold values and detecting motion based upon combinations of the comparisons when the factors exceed the threshold value.