A method and apparatus of generating a three-dimensional (3D) image are provided. The method of generating a 3D image involves acquiring a plurality of images of a 3D object with a camera, calculating pose information of the plurality of images based on pose data for each of the plurality of images measured by an inertial measurement unit, and generating a 3D image corresponding to the 3D object based on the pose information.