Systems and methods for controlling the operation of an autonomous vehicle are disclosed herein. One embodiment performs traffic light detection at an intersection using a sensor-based traffic light detector to produce a sensor-based detection output, the sensor-based detection output having an associated first confidence level; performs traffic light detection at the intersection using a vehicle-to-infrastructure-based (V2I-based) traffic light detector to produce a V2I-based detection output, the V2I-based detection output having an associated second confidence level; performs one of (1) selecting as a final traffic-light-detection output whichever of the sensor-based detection output and the V2I-based detection output has a higher associated confidence level and (2) generating the final traffic-light-detection output by fusing the sensor-based detection output and the V2I-based detection output using a first learning-based classifier; and controls the operation of the autonomous vehicle based, at least in part, on the final traffic-light-detection output.