A method, system and computer program product for compensating the environmental motion in handheld devices. A sensor unit is affixed to an object in the environment to detect and measure environmental motion. Upon measuring any detected environmental motion, the sensor unit transmits a value corresponding to the measured environmental motion to one or more handheld devices. Alternatively, the sensor unit may transmit the value corresponding to the measured environmental motion to a unit configured to retransmit the value to one or more handheld devices. Upon receiving the value corresponding to the measured environmental motion, the handheld device cancels this environmental motion from the motion it measured thereby taking into consideration only the motion inputted by the user of the handheld device.