A radiation therapy delivery system (10) includes an ultrasound imaging unit (26), a radiation therapy delivery mechanism (12, 56, 70, 88), a plurality of fiducials (22, 90) located internal to the subject, an image fusion unit (40), and a delivery evaluation unit (38). The ultrasound imaging unit (26) includes a transducer (30) that emits ultrasonic sound waves to image in real-time an anatomic portion of a subject (16) in a first coordinate system. The radiation therapy delivery mechanism (12, 56, 70, 88) delivers amounts of therapeutic radiation in the anatomic portion of the subject in a second coordinate system. The fiducials (22, 90) include implants or a trans-rectal ultrasound probe (80). The image fusion unit (40) registers locations of the plurality of fiducials to at least one of the first and the second coordinate system and tracks the locations of the fiducials in real-time. The delivery evaluation unit (38) identifies locations and the amounts of delivered therapeutic radiation relative to the imaged real-time anatomic portion of the subject.