A method of fabricating a CMOS semiconductor device is provided, which decreases the number of necessary photolithography processes for forming the LDD and pocket structures. A first pair of doped regions of a first conductivity type are formed in a first section of a semiconductor substrate and a second pair of doped regions of the first conductivity type are formed in a second section thereof. Then, a third pair of doped regions of a second conductivity type are formed in the first pair of doped regions and a fourth pair of doped regions of the second conductivity type are formed in the second pair of doped regions. Thereafter, an impurity of the second conductivity type is selectively ion-implanted into the first section while covering the second section with a mask, thereby forming a fifth pair of doped regions of the second conductivity type from the first pair of remaining doped regions. Further, another impurity of the first conductivity type is selectively ion-implanted into the first section while the second section is covered with the mask, thereby forming a sixth pair of doped regions of the first conductivity type from the second pair of doped regions. The channeling effect of an implanted impurity may be utilized, where a pair of doped regions in one of the first and second sections are formed deeper than another pair of doped regions in the other.