Provided is an operating method of an imaging system, the operating method including illuminating, by a light illuminator, light onto a subject from a light source, acquiring, by an optical detector, a two-dimensional projection image on the subject, generating a first projection matrix between three-dimensional coordinates of the subject and two-dimensional coordinates of the projection image, generating a three-dimensional first volume image on the subject on the basis of the first projection matrix and the projection image, generating a two-dimensional digitally reconstructed radiograph (DRR) on the subject from the first volume image, matching the projection image and the DRR, and updating the first projection matrix on the basis of a matched result to generate a second projection matrix, and generating a three-dimensional second volume image on the subject on the basis of the second projection matrix and the projection image.