Systems, methods, and computer-readable media are provided for performing image segmentation with depth filtering. In some examples, a method can include obtaining a frame capturing a scene: generating, based on the frame, a first segmentation map including a target segmentation mask identifying a target of interest and one or more background masks identifying one or more background regions of the frame; and generating a second segmentation map including the first segmentation map with the one or more background masks filtered out, the one or more background masks being filtered from the first segmentation map based on a depth map associated with the frame.