The semiconductor device may include an active pattern provided on a substrate and a source/drain pattern on the active pattern. The source/drain pattern may include a bottom surface in contact with a top surface of the active pattern. The semiconductor device may further include a channel pattern connected to the source/drain pattern, a gate electrode extended to cross the channel pattern, and a fence insulating layer extended from a side surface of the active pattern to a lower side surface of the source/drain pattern. A pair of middle insulating patterns may be at both sides of the bottom surface of the source/drain pattern and between the active pattern and the source/drain pattern in contact with an inner side surface of the fence insulating layer.