The present invention relates to an apparatus for rejecting images in a receiver.The apparatus of the present invention relates to an apparatus for rejecting image signals in a receiver of a direct conversion structure and comprises a signal mismatch compensation unit configured to detect gain error and phase error between an In-phase (I) signal and a Quadrature (Q) signal received through the receiver, to reject image signals existing in the I and Q signals, and to output a result. The signal mismatch compensation unit detects the gain error and the phase error using an adaptive step method of reducing the step size of the gain error and the phase error step by step whenever the gain error and the phase error are converged.According to the present invention, high image rejection ratio is achieved and the adaptation time taken to obtain a high image rejection ratio is reduced simultaneously. Further, a bad influence of the DC offset on the image rejection ratio can be prevented by removing DC offset signals in a digital structure, accordingly, error can be accurately estimated.