A method of preparing a nitrile rubber composition having modulus and excellent physical properties and abrasion resistance is disclosed. This composition is prepared by kneading at 120.degree.-160.degree. C. (a) a nitrile rubber, (b) an .alpha.,.beta.-ethylenically unsaturated carboxylic acid, (c) a divalent metal compound, (d) a secondary aryl amine, cooling the kneaded mass below 100.degree. C., adding to the kneaded mass (e) an organic peroxide, and heating and curing the resulting mixture at 110.degree.-180.degree. C. The addition of the secondary aryl amine considerably improves the physical properties of the composition. These compositions have not a tendency of scorching and are superior in the workability as compared with conventional carboxylated nitrile rubber compositions produced by emulsion polymerization.