A method for processing video data includes performing by one or more processors and/or circuits in a video processing device, the one or more processors and/or circuits including a video scaler, a memory, and a scaler engine, functions including receiving a video image by the video processing device. The functions also include determining whether the video scaler requires less memory bandwidth to scale the video image before writing the video image to the memory or after reading the video image from the memory, and scaling the video image based on the determination. If the video scaler requires less memory bandwidth to scale the video image before writing the video image to the memory, performing by the one or more processors and/or circuits scaling of the video image in the video scaler using a video input clock of the video scaler to generate a first scaled video image.