This invention relates to a system that adaptively compensates for subject motion in real-time in an imaging system. An object orientation marker, preferably a retro-grate reflector (RGR), is placed on an organ of interest of a patient during a scan, such as an MRI scan. The marker allows measuring the six degrees of freedom or “pose” required to track motion of the organ of interest. A detector, preferably a camera, observes the marker and continuously extracts its pose. The pose from the camera is sent to the scanner via an RGR processing computer and a scanner control and processing computer, allowing for continuous correction of scan planes and position (in real-time) for motion of the patient. This invention also provides for internal calibration and for co-registration over time of the scanner's and tracking system's reference frames to compensate for drift and other inaccuracies that may arise over time.