A vehicular driver assistance system includes an attachment element attached at an in-cabin surface of a windshield of the vehicle. The attachment element includes a mirror mounting element and an interior rearview mirror assembly includes a mirror head and a mirror support. The mirror support includes a mirror mount that is configured to mount the interior rearview mirror assembly to the mirror mounting element. A structure is attached at the vehicle windshield local to the attachment element and the structure accommodates a forward facing camera and, with the structure attached at the vehicle windshield, the forward facing camera views through the vehicle windshield. The forward facing camera may include a component of at least one of (i) an automatic headlamp control system, (ii) a collision avoidance system, (iii) a windshield fogging detection system and (iv) a rain sensing system.