The disclosure relates to a hardware-in-the-loop simulation system and method for computer vision. An embodiment of the disclosed system comprises a software simulation and a hardware simulation. The software simulation includes a virtual scene and an observed object that are generated by virtual reality software. The virtual scene images are obtained at different viewpoints. The hardware simulation includes the virtual scene images being projected onto a screen by a projector, wherein the projected scene images are shot by a camera, and where in the direction of the camera is controlled by a pan-tilt.