A high voltage semiconductor device includes a semiconductor substrate, an isolation structure, a gate oxide layer, and a gate structure. The semiconductor substrate includes a channel region, and at least a part of the isolation structure is disposed in the semiconductor substrate and surrounds the channel region. The gate oxide layer is disposed on the semiconductor substrate, and the gate oxide layer includes a first portion and a second portion. The second portion is disposed at two opposite sides of the first portion in a horizontal direction, and a thickness of the first portion is greater than a thickness of the second portion. The gate structure is disposed on the gate oxide layer and the isolation structure.