A high voltage semiconductor device comprises a substrate, a well, a gate structure, and a source/drain structure in a grade region in a well in the substrate. The gate structure is disposed on the substrate with a portion vertically down into a trench in the well in the substrate and has a relatively small size. The method of fabricating the high voltage semiconductor device comprises forming a first trench for an STI structure and a second trench for a gate structure, depositing an oxide layer on the substrate to fill the first and the second trenches, wherein a void is formed in the second trench, performing a photolithography and etching process to remove a portion of the oxide layer in the second trench, and forming a gate on the gate dielectric layer in the second trench.