A system and method is disclosed for management of sample data to enable video rate anti-aliasing convolution. Sample data may be moved simultaneously from a sample buffer to a bin scanline cache and from the bin scanline cache to an array of N2 processor—memory units (e.g. 25 for N=5). Pixel data may be convolved from an N×N sample bin array that may be approximately centered on the pixel location. Since each sample bin contains Ns/b samples, Ns/b×N2 samples may be filtered for each pixel (e.g. 400 for N=5 and Ns/b=16). Each processor—memory unit convolves the sample data for one sample bin in the N×N sample bin array and supports a variety of filter functions. Pixel data may be output to a real time video data stream.