A process for heat-treating of rare earth permanent magnets chiefly of the Nd-Fe-B system. In the permanent magnet easily oxidized in the open air, an alloy thereof is pulverized, orientated compression-molded in a magnetic field or compression-molded in a non-magnetic field, and sintered at 900° to 1200°C, then ground and polished into a practical shape, subjected to solution heat treatment at 900° to 1200°C in an atmosphere of oxygen and/or nitrogen, or aged at 300° to 900 °C, so that an oxide and/or a nitride film of 0.001 to 10 µm in thickness is formed on the surface of the magnet in order to prevent the oxidation and to remove work strain.