A visual line direction sensing device communicably connected with a head-up display device, which projects an image on a screen disposed ahead of a target, includes an image capture unit capturing a target image from a front side of the target, an acquisition unit acquiring adjusted position information indicative of an image projection position by an adjustment unit of the head-up display device, an estimation unit estimating positions of pupils of the target based on the adjusted position information acquired by the acquisition unit, a setup unit setting an inspection range of the target image captured by the image capture unit with reference to the positions of the pupils estimated by the estimation unit, and a calculator calculating a visual line direction of the target based on a status of the pupils included in the inspection range of the target image set by the setup unit.