The invention provides a monocular and binocular vision system for target detection and pose measurement constructed by a PTZ camera, comprising a PTZ camera and a control system, a precision displacement sliding table and a driving device, and an image processing system. The system controls the PTZ camera to make linear motion on the precision linear sliding table, and only a single camera can realize the function of binocular vision system. Based on the model of object detection based on depth learning, the object detection module is constructed. When the Zoom of PTZ camera is kept unchanged, two images of any scene captured under differnet Pan-Tilt are used to calibrate all the internal parameters of PTZ camera in real time. A 3D pose measurement module is constructed by using the proposed algorithm to realize high-precision matching of feature points in color distorted images. Therefore, the monocular and binocular vision system uses only one PTZ camera, which can realize the functions of the monocular camera, such as object detection and tracking, etc. The binocular system functions, such as binocular three-dimensional measurement, robot vision guidance and so on can also be achieved.