An interior rearview mirror system includes an interior rearview mirror assembly and a compass sensor. A physical mount mounts the compass sensor at the reflective element whereby the compass sensor adjusts in tandem with adjustment of the reflective element. A control receives at least one signal indicative of at least one magnetic field sensed by the compass sensor. The control determines a directional heading of the vehicle based on the signal or signals and generates a signal indicative of the directional heading of the vehicle. The control may automatically compensate for adjustment of the compass sensor in tandem with adjustment of the reflective element.