An anti-fuse structure that can be programmed at low voltage and current and which potentially consumes very little chip spaces and can be formed interstitially between elements spaced by a minimum lithographic feature size is formed on a composite substrate such as a silicon-on-insulator wafer by etching a contact through an insulator to a support semiconductor layer, preferably in combination with formation of a capacitor-like structure reaching to or into the support layer. The anti-fuse may be programmed either by the selected location of conductor formation and/or damaging a dielectric of the capacitor-like structure. An insulating collar is used to surround a portion of either the conductor or the capacitor-like structure to confine damage to the desired location. Heating effects voltage and noise due to programming currents are effectively isolated to the bulk silicon layer, permitting programming during normal operation of the device. Thus the potential for self-repair without interruption of operation is realized.