The invention discloses a ballast swing backfilling device which comprises a front end hanging bracket seat mounted at a main frame, and a rotary hanging bracket seat, wherein a rack is mounted at the rotary hanging bracket seat by a rotary hanging bracket; the front end hanging bracket seat is provided with a rail bracket; the rail bracket is placed at a rail; a rolling bearing is mounted on a front end hanging bracket, is placed on the rail, and can roll along the rail; the front part of the rack is mounted at the lower end of the front end hanging bracket; one end of a swing oil cylinder is mounted on the front end hanging bracket seat; the other end of the swing oil cylinder is mounted at the rack; and a belt conveyor is mounted at the rack. According to the ballast swing backfilling device, the swing speed of the swing belt conveyor, and the pause time and angle at the two ends of the rail can be set according to line conditions; a jetting distance from a ballast backfilling point to the swing belt conveyor can be controlled by adjusting the rotating speed of a drive pulley; and therefore, a distribution position of backfilling ballast can be controlled, the line recovery time can be shortened effectively, and the cleaning operation efficiency can be improved.