A system and method are presented for the automatic filtering of test utterance mismatches in automatic speech recognition (ASR) systems. Test data are evaluated for match between audio and text in a language-independent manner. Utterances having mismatch are identified and isolated for either removal or manual verification to prevent incorrect measurements of the ASR system performance. In an embodiment, contiguous stretches of low probabilities in every utterance are searched for and removed. Such segments may be intra-word or cross-word. In another embodiment, scores may be determined using log DNN probability for every word in each utterance. Words may be sorted in the order of the scores and those utterances containing the least word scores are removed.