A gate structure in a semiconductor device includes a tunnel insulation layer disposed on a substrate, a first charge trapping layer disposed on the tunnel insulation layer, a second charge trapping layer disposed on the first charge trapping layer, a dielectric layer disposed to cover the second charge trapping layer, and a conductive layer pattern disposed on the dielectric layer. The first charge trapping layer includes charge trapping sites for storing charges therein. The second charge trapping layer includes nanocrystals. The semiconductor device including the gate structure may have a sufficiently wide programming/erasing window and an improved data retention capability.