Embodiments disclosed herein provide systems, methods, and computer readable media for replacing a video background in real-time. The video comprises a plurality of image frames. In a particular embodiment, a method provides generating a range image from a subject image frame of the plurality of image frames. The range image indicates pixel distances from a plenoptic camera that captured the plurality of image frames. The method further provides identifying background pixels that represent a background portion of the subject image frame based on the range image and replacing the background pixels with replacement background pixels in the subject image frame.