An ultrasound imaging apparatus includes an image generator configured to scan an object to obtain an ultrasound image; a neural network configured to generate a virtual ultrasound image based on matching medical images of different modalities; an image converting part configured to convert a medical image of a different modality, previously obtained by scanning the object, into the virtual ultrasound image by using the neural network; and a matching part configured to determine a position of a virtual probe based on the ultrasound image and the virtual ultrasound image, and configured to match the ultrasound image with the medical image that corresponds to the determined position of the virtual probe.