A PET scanner (20, 22, 24, 26) generates a plurality of time stamped lines of response (LORs). A motion detector (30) detects a motion state, such as motion phase or motion amplitude, of the subject during acquisition of each of the LORs. A sorting module (32) sorts the LORs by motion state and a reconstruction processor (36) reconstructs the LORs into high spatial, low temporal resolution images in the corresponding motion states. A motion estimator module (40) determines a motion transform which transforms the LORs into a common motion state. A reconstruction module (50) reconstructs the motion corrected LORs into a static image or dynamic images, a series of high temporal resolution, high spatial resolution images.