Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using traffic light viewsheds. One of the methods includes obtaining, by an on-board computer system of vehicle, viewshed data comprising a plurality of viewsheds for a plurality of respective traffic lights. An indication of a candidate driving plan that uses input from a traffic light detection of a particular traffic light having a viewshed is received. If a current location of the vehicle is within the particular viewshed for the particular traffic light, execution of the candidate driving plan is conditioned based on whether the current location of the vehicle is within the particular viewshed for the particular traffic light.