A method for eye tracking in a head-mountable device (HMD) includes determining at least one object within a three-dimensional (3D) extended reality (XR) environment as an eye tracking calibration point and determining a 3D location of the eye tracking calibration point within the XR environment. The method also includes detecting a gaze point of a user of the HMD and comparing the detected gaze point to an area of the XR environment that includes the 3D location of the eye tracking calibration point. The method further includes, in response to determining that the user is looking at the eye tracking calibration point based on the detected gaze point being within the area, calibrating, using a processor, the HMD to correct a difference between the eye tracking calibration point and the detected gaze point. In addition, the method includes, in response to determining that the user is not looking at the eye tracking calibration point based on the detected gaze point being outside of the area, maintaining an existing calibration of the HMD.