A cleaning device for a recording and/or reproducing apparatus including a rotary head device having a drum carrying a magnetic head, with a magnetic tape being used as a recording medium. The cleaning device includes a cleaning member abutted against the magnetic head of the rotary head device for cleaning the magnetic head and an actuating mechanism for causing the cleaning member to be abutted against the magnetic head in the rotating state of the rotary head device. The actuating mechanism causes the cleaning member to be abutted against the magnetic head in the rotating state of the rotary drum prior to the loading of the magnetic tape on the rotary head device by the tape loading mechanism, and releases abutment of the cleaning member against the magnetic head when loading the magnetic tape by the tape loading mechanism on the rotary head device. The magnetic head is cleaned each time the magnetic tape is loaded.