A hot spot detection system for automatically segmenting and quantifying hot spots in functional images includes a segmentation unit (76) to segment an anatomical image representation (72) into regions corresponding to anatomical structures of a subject. A hot spot detection unit (90) detects regions of high uptake from a functional second image representation (74). The regions of high tracer uptake indicate high metabolic activity which maybe caused by potentially hazardous tumor lesions or other malignant processes. However, a number of normally functioning organs uptake high amounts of imaging tracer, particularly FDG. Therefore, a suppression unit (102) suppresses regions of high tracer uptake in the functional second image representation based on the results of a classification unit (101). The classification unit classifies the regions of high tracer uptake according to their position relative to the anatomical structures segmented from the anatomical first image representation. The remaining un-suppressed regions of high uptake are identified by an identification unit (106) as one of potential lesion and non-potential lesion.