The present technology relates to an image processing device, a method of image processing, and a surgical microscope that can detect and report a dangerous condition on the basis of a tomographic image during eye surgery. An image processing device includes: a dangerous condition detection unit configured to detect a dangerous condition on the basis of a tomographic image of an eye acquired during surgery of the eye; and a control information generation unit configured to generate and output control information used to manage the detected dangerous condition. The present technology is applicable to, for example, a surgical system used for eye surgery or other surgical procedures.