A method of fabricating a semiconductor with self-aligned spacer includes providing a substrate. At least two gate structures are disposed on the substrate. The substrate between two gate structures is exposed. A silicon oxide layer is formed to cover the exposed substrate. A nitride-containing material layer covers each gate structure and silicon oxide layer. Later, the nitride-containing material layer is etched to form a first self-aligned spacer on a sidewall of each gate structure and part of the silicon oxide layer is exposed, wherein the sidewalls are opposed to each other. Then, the exposed silicon oxide layer is removed to form a second self-aligned spacer. The first self-aligned spacer and the second self-aligned spacer cooperatively define a recess on the substrate. Finally, a contact plug is formed in the recess.