A negative resist composition comprising (A) an alkali-soluble polymer, (B) a cross-linking agent forming cross-links between molecules of the alkali-soluble polymer (A) under the action of an acid and (C) a specified acid generator, which can satisfy all of performance requirements concerning sensitivity, resolution, pattern profile and line-edge roughness in the pattern formation by irradiation with electron beams or X-rays.