A device for cleaning the tip portion and the side surfaces of a probe has two or more different kinds of intermediate sheets affixed to a substrate in a side-by-side relationship with respect to each other. One of these intermediate sheets is an elastic sheet having an elastic property. Another intermediate sheet of a different kind is a plastic sheet which is less elastic. The tip portion of a probe is cleaned by a polishing layer affixed to the elastic sheet. The side surfaces of the probe are cleaned by a polishing layer affixed to the plastic sheet. A porous foamed sheet having openings on its surface and having air bubbles inside may be used as the elastic sheet. A polishing layer is formed on the surface of the foamed sheet, having a porous surface with openings corresponding to the openings on the foamed sheet. A spacer may be provided between the substrate and the plastic sheet for adjusting the heights of the polishing layers formed over the intermediate sheets.