A cleaning medium for magnetic recording devices comprises a non-magnetic substrate, a lower coating layer, which is overlaid upon the non-magnetic substrate and primarily contains a binder and non-magnetic inorganic particles dispersed in the binder, and a cleaning layer, which is overlaid upon the lower coating layer and contains a binder and inorganic particles dispersed in the binder and at least containing ferromagnetic particles. An at least 50% by weight portion of the non-magnetic inorganic particles, which are contained in the lower coating layer, is constituted of granular inorganic particles, which have a mean particle diameter of at most 0.08 .mu.m, or acicular inorganic particles, which have a mean longer axis length falling within the range of 0.05 .mu.m to 0.3 .mu.m and an acicular ratio falling within the range of 3 to 20. The cleaning medium removes dirt from magnetic heads with high cleaning power, such that the wear of the magnetic heads may be small and no scratch may occur on the cleaned magnetic heads.