A depth image acquiring device is provided, which includes at least one projecting device and at least one image sensing device. The projecting device projects a projection pattern to an object. The image sensing device senses a real image. In addition, the projecting device also serves as a virtual image sensing device. The depth image acquiring device generates a disparity image by matching three sets of dual-images formed by two real images and one virtual image, and generates a depth image according to the disparity image. In addition, the depth image acquiring device also generates a depth image by matching two real images, or a virtual image and a real image without verification.