A code region forming part of a computer program is modified during execution of the computer program by a plurality of threads. In one aspect, identical modification instructions are provided to each thread for modifying a site in the code region having a desirable idempotent atomic modification, and the modification instructions direct each thread to make the desirable idempotent atomic modification. In another aspect, a thread is selected to modify the code region, each thread other than the selected thread is directed to execute an alternative execution path that generates output identical to the output of the code region after the code region has been modified, and, responsive to directing each thread other than the selected thread, the selected thread is directed to modify the code region.