A method for neutral pixel detection using color space feature vectors wherein one color space coordinate represents lightness is provided. The method includes the following steps: a) receiving an input image represented in a first color space; b) converting the input image to a second color space wherein one coordinate represents lightness; c) selecting a pixel in the second color space representation to be classified; d) computing second color space feature vectors associated with the selected pixel; and e) classifying the selected pixel between neutral and color classes based on the values computed for the second color space feature vectors. Typically, the input image is processed using a smoothing filter to create a smoothed input image prior to the conversion to the second color space. The method can be adapted to page processing or strip processing schemes with respect to the input image.