An augmented reality system recognizes objects in a user's environment and operates an electronic contact lens based on the recognition. The electronic contact lens includes an integrated femtoimager that captures images corresponding to the user's gaze direction. The augmented reality system recognizes objects in the images and generates visual information relevant to the recognized objects that is presented using a femtoprojector integrated with the electronic contact lens. The visual information may include virtual control elements that the user can interact with to control smart devices. The augmented reality system can also configure various calibration parameters of the electronic contact lens based on a recognized environment associated with the recognized objects.