Methods and systems for detecting defects on a wafer using adaptive local thresholding and color filtering are provided. One method includes determining local statistics of pixels in output for a wafer generated using an inspection system, determining which of the pixels are outliers based on the local statistics, and comparing the outliers to the pixels surrounding the outliers to identify the outliers that do not belong to a cluster of outliers as defect candidates. The method also includes determining a value for a difference in color between the pixels of the defect candidates and the pixels surrounding the defect candidates. The method further includes identifying the defect candidates that have a value for the difference in color greater than or equal to a predetermined value as nuisance defects and the defect candidates that have a value for the difference in color less than the predetermined value as real defects.