The disclosed technology generally relates to semiconductor devices, and more specifically to a semiconductor device having a high aspect ratio channel layer. In one aspect, semiconductor device includes a semiconductor substrate having formed thereon a dielectric isolation layer having an opening formed therethrough. The semiconductor device additionally includes a filling isolation structure having a portion formed in the opening and a portion protruding above the dielectric isolation layer, wherein the filling isolation structure comprises a dielectric filling layer. The semiconductor device additionally includes a dielectric layer formed on the dielectric isolation layer, wherein the dielectric layer and the dielectric filling layer have top surfaces that are substantially co-planar to form a common top surface. The semiconductor device further includes a first vertical channel layer laterally interposed between and in contact with the dielectric layer and the dielectric filling layer at a first side of the dielectric filling layer, wherein the first vertical channel layer extends above the common top surface.