An actinic ray-sensitive or radiation-sensitive resin composition contains (A) a resin having a polarity that increases by an action of an acid, (B) a photoacid generator, (P) an amine oxide, and (D) an acid diffusion control agent (provided that acid diffusion control agents corresponding to the amine oxide are excluded), in which a content of the amine oxide (P) is from 0.01 ppm to 1,000 ppm with respect to a total mass of the actinic ray-sensitive or radiation-sensitive resin composition, and a mass ratio of the acid diffusion control agent (D) to the amine oxide (P) is more than 1 and 10,000 or less.