Systems and techniques that facilitate real-time and/or offline de-identification of facial regions from regular and/or occluded color video streams obtained during diagnostic medical procedures are provided. A detection component can generate a bounding box substantially around a person in a frame of a video stream, can generate a heatmap showing key points or anatomical masks of the person based on the bounding box, and can localize a face or facial region of the person based on the key points or anatomical masks. An anonymization component can anonymize pixels in the frame that correspond to the face or facial region. A tracking component can track the face or facial region in a subsequent frame based on a structural similarity index between the frame and the subsequent frame being above a threshold. If the structural similarity index between the frame and the subsequent frame is above the threshold, the tracking component can track the face or facial region in the subsequent frame without having the detection component generate a bounding box or a heatmap in the subsequent frame, and the anonymization component can anonymize pixels in the subsequent frame corresponding to the tracked face or facial region.