An apparatus and method for determining a block size assignment for a block of pixels is claimed. A mean for a given block of pixels and a neighborhood mean for adjacent blocks of pixels is determined. A contrast ratio is then determined, as a function of the mean and the neighborhood mean. A variance threshold is determined as a function of the contrast ratio. Then, various sub-block sizes within the block of pixels is determined as a function of the variance threshold. As such, a pixel block is considered in the context of adjacent pixel blocks rather than in isolation, which narrows the spread of block variances and allows for a more robust subdivision of pixel blocks.