A semiconductor device includes a silicon substrate (1), a pair of isolating insulation films (9), a channel region (2), a pair of source/drain regions (3), a pair of silicon oxide films (4) formed on an upper surface of the silicon substrate (1) so as to overlie the source/drain regions (3), and a gate structure (8) formed in a first recess defined by the upper surface of the silicon substrate (1) over the channel region (2) and side surfaces of the pair of silicon oxide films (4). The gate structure (8) includes a gate oxide film (5) formed on the upper surface of the silicon substrate (1), a pair of silicon oxide films (6) formed on lower part of the side surfaces of the pair of silicon oxide films (4), and a metal film (7) filling a second recess surrounded by upper part of the side surfaces of the silicon oxide films (4), the silicon oxide films (6) and the gate oxide film (5). A method of manufacturing the semiconductor device is provided which attains reduction in gate length without the decrease in driving capability to accomplish the increase in operating speed.