A semiconductor device includes a silicon-on-insulator (SOI) substrate having a buried oxide (BOX) layer, and a plurality of semiconductor fins formed on the BOX layer. The plurality of semiconductor fins include at least one pair of fins defining a BOX region therebetween. Gate lines are formed on the SOI substrate and extend across the plurality of semiconductor fins. Each gate line initially includes a dummy gate and a hardmask. A high dielectric (high-k) layer is formed on the hardmask and the BOX regions. At least one spacer is formed on each gate line such that the high-k layer is disposed between the spacer and the hardmask. A replacement gate process replaces the hardmask and the dummy gate with a metal gate. The high-k layer is ultimately removed from the gate line, while the high-k layer remains on the BOX region.