Disclosed is a mechanism for limiting Intersymbol Interference (ISI) when measuring uncorrelated jitter in a test and measurement system. A waveform is obtained that describes a signal. Such waveform may be obtained from memory. A processor then extracts a signal impulse response from the waveform. The processor selects a window function based on a shape of the signal impulse response. Further, the processor applies the window function to the signal impulse response to remove ISI outside a window of the window function while measuring waveform jitter. The window function may be applied by applying the window function to the signal impulse response to obtain a target impulse response. A linear equalizer is then generated that results in the target impulse response when convolved with the signal impulse response. The linear equalizer is then applied to the waveform to limit ISI for jitter measurement.