A semiconductor structure is provided that includes a material stack including an epitaxially grown semiconductor layer on a base semiconductor layer, a dielectric layer on the epitaxially grown semiconductor layer, and an upper semiconductor layer present on the dielectric layer. A capacitor is present extending from the upper semiconductor layer through the dielectric layer into contact with the epitaxially grown semiconductor layer. The capacitor includes a node dielectric present on the sidewalls of the trench and an upper electrode filling at least a portion of the trench. A substrate contact is present in a contact trench extending from the upper semiconductor layer through the dielectric layer and the epitaxially semiconductor layer to a doped region of the base semiconductor layer. A substrate contact is also provided that contacts the base semiconductor layer through the sidewall of a trench. Methods for forming the above-described structures are also provided.