Apparatus for electrical flash butt welding the ends of metal sheet or strip including a pair of flat faced parallel welding electrodes extending across the full width of the metal sheet or strip, one on each side thereof, and an electrode cleaning device comprising a support assembly having mounted thereon an extendable electrode cleaning means including a cleaning head having a plurality of upward extending cleaning members and a plurality of downwardly extending cleaning members the cleaning head adapted to simultaneously remove dirt and contaminants from the contact surfaces of opposed welding electrodes.