A semiconductor device includes first and second trenches in respective first and second regions in a substrate, a first isolation structure having a first inner wall oxide pattern, a first liner, and a first filling insulation pattern sequentially stacked I the first trench, a second isolation structure having a second inner wall oxide pattern, a second liner, and a second filling insulation pattern sequentially stacked I the second trench, a first gate structure having a first high-k dielectric pattern, a first P-type metal pattern, and a first N-type metal pattern sequentially stacked on the first region, and a second gate structure having a second high-k dielectric pattern and a second N-type metal pattern sequentially stacked on the second region, wherein the first and second liners protrude above upper surfaces of the first and second inner wall oxide patterns and the first and second filling insulation patterns, respectively.