Techniques for aligning a virtual object with a physical object in an Augmented Reality (AR) or Virtual Reality (VR) application are described. An electronic peripheral includes a first inertial measurement unit (“IMU”). A head mounted display includes a second IMU. An estimated attitude for the electronic peripheral is generated using data from the first IMU. An estimated attitude for the head mounted display is generated using data from the second IMU. An orientation of a virtual object is determined based on the estimated first and second attitudes, such that the virtual object is aligned with an object in a user's physical environment when the virtual object is displayed to the user. The virtual object is displayed on the head mounted display.