Systems and methods are provided in which devices that are employed during a medical procedure are adaptively configured during the medical procedure, based on input or feedback that is associated with the current state, phase or context of the medical procedure. In some example embodiments, the input is obtained via the identification of one or more medical instruments present within a region of interest, and this input may be employed to determine configuration parameters for configuring the device. In other example embodiments, the input may be based on the image-based detection of a measure associated with the phase or context of the medical procedure, and this input may be employed to adaptively control the device based on the inferred context or phase of the medical procedure. In other embodiments, images from one imaging modality may be employed to adaptively switch to another imaging modality.