In accordance with an embodiment, a pattern evaluation system includes an image acquisition unit, a plurality of image processing units, and a control unit which controls the plurality of image processing units. The image acquisition unit loads a series of images of a pattern to be evaluated. The images are acquired at a first speed. The plurality of image processing units process the series of images at a second speed and then output a result of the evaluation of the pattern to be evaluated. The control unit acquires the first and second speeds, estimates the number of the image processing units which allow the time for acquiring the series of images to be substantially the same as the time for processing the series of images, and allocates the estimated image processing units to the processing of the series of images.