Aspects of the disclosure relate to identifying construction objects. As an example, an image captured by a camera associated with a vehicle as the vehicle is driven along a roadway may be received. This image may be converted into a first channel corresponding to an average brightness contribution from red, blue and green channels of the image. The image may also be converted into a second channel corresponding to a contribution of a color from the red and the green channels of the image. A template may then be used to identify a region of the image corresponding to a potential construction object from the first channel and the second channel.