A permanent magnet which contains R, T and B as main ingredients wherein R is Y or a rare earth element and T is Fe or Fe and Co and has a primary phase of R.sub.2 T.sub.14 B is produced by compacting a mixture of 60 to 95 wt % of a primary phase-forming master alloy and a grain boundary phase-forming master alloy both in powder form and sintering the compact. The primary phase-forming master alloy has columnar crystal grains of R.sub.2 T.sub.14 B with a mean grain size of 3-50 .mu.m and grain boundaries of an R rich phase and contains 26-32 wt % of R. The grain boundary phase-forming master alloy is a crystalline alloy consisting essentially of 32-60 wt % of R and the balance of Co or Co and Fe. In anther form, a permanent magnet which contains R, T and B as main ingredients wherein R is yttrium or a rare earth element, T is Fe or Fe+Co/Ni and has a primary phase of R.sub.2 T.sub.14 B is produced by compacting a mixture of a primary phase-forming master alloy and a grain boundary-forming master alloy both in powder form and sintering the compact. The primary phase-forming master alloy has a primary phase of R.sub.2 T.sub.14 B and grain boundaries of an R rich phase. The grain boundary-forming master alloy contains 40-65 wt % of R, 30-60 wt % of Fe, Co or Ni and 1-12 wt % of Sn, In or Ga.