The structure of a semiconductor device with passivation layers on active regions of FET devices and a method of fabricating the semiconductor device are disclosed. The semiconductor device includes a substrate, first and second source/drain (S/D) regions disposed on the substrate, nanostructured channel regions disposed between the first and second S/D regions, a passivation layer, and a nanosheet (NS) structure wrapped around the nanostructured channel regions. Each of the S/D regions have a stack of first and second semiconductor layers arranged in an alternating configuration and an epitaxial region disposed on the stack of first and second semiconductor layers. A first portion of the passivation layer is disposed between the epitaxial region and the stack of first and second semiconductor layers and a second portion of the passivation layer is disposed on sidewalls of the nanostructured channel regions.