A ranging system including a first aperture stop, a second aperture stop, an image sensor and a processing unit is provided. The first aperture stop includes a sheet of IR-cut filter material. The second aperture stop includes a sheet of opaque material. The image sensor receives light passing through the first and second aperture stops to output an image frame. The processing unit respectively calculates a convolution between a first color subframe of the image frame and a plurality of first blur kernels to generate a plurality of blurred first frames, respectively calculates a convolution between a second color subframe of the image frame and a plurality of second blur kernels to generate a plurality of blurred second frames, and performs a best matching between the blurred first frames and the blurred second frames.