This invention relates to a system that adaptively compensates for subject motion in real-time in an imaging system. An object orientation marker (30), preferably a retro-grate reflector (RGR), is placed on the head or other body organ of interest of a patient (P) during a scan, such as an MRI scan. The marker (30) makes it possible to measure the six degrees of freedom (x, y, and z-translations, and pitch, yaw, and roll), or “pose”, required to track motion of the organ of interest. A detector, preferably a camera (40), observes the marker (30) and continuously extracts its pose. The pose from the camera (40) is sent to the scanner (120) via an RGR processing computer (50) and a scanner control and processing computer (100), allowing for continuous correction of scan planes and position (in real-time) for motion of the patient (P). This invention also provides for internal calibration and for co-registration over time of the scanner's and tracking system's reference frames to compensate for drift and other inaccuracies that may arise over time.