A rare earth permanent magnet is prepared by disposing a powdered metal alloy containing at least 70 vol % of an intermetallic compound phase on a sintered body of R—Fe—B system, and heating the sintered body having the powder disposed on its surface below the sintering temperature of the sintered body in vacuum or in an inert gas for diffusion treatment. The advantages include efficient productivity, excellent magnetic performance, a minimal or zero amount of Tb or Dy used, an increased coercive force, and a minimized decline of remanence.