A capacitor in a semiconductor substrate employs a conductive through-substrate via (TSV) as an inner electrode and a columnar doped semiconductor region as an outer electrode. The capacitor provides a large decoupling capacitance in a small area, and does not impact circuit density or a Si3D structural design. Additional conductive TSV's can be provided in the semiconductor substrate to provide electrical connection for power supplies and signal transmission therethrough. The capacitor has a lower inductance than a conventional array of capacitors having comparable capacitance, thereby enabling reduction of high frequency noise in the power supply system of stacked semiconductor chips.