A method for producing a nanocrystalline alloy wherein an amorphous alloy is heat-treated by keeping the temperature at a first heat treatment temperature higher than the crystallization temperature of the amorphous alloy for 0 to less than 5 minutes, and is cooled to room temperature at a cooling rate of 20.degree. C./min or more at least until the temperature falls to 400.degree. C. The amorphous alloy subjected to the first heat treatment may be further heat-treated at a second heat treatment temperature not higher than 500.degree. C. and lower than the first heat treatment temperature while applying a magnetic field. The nanocrystalline alloy produced by the method of the invention has a extremely high specific initial permeability as compared with the conventional nanocrystalline alloy, and is suitable for use in magnetic core of transformers, choke coils, etc.