Disclosed are an apparatus and method with video processing. A computing apparatus includes one or more processors and storage storing instructions configured to, when executed by the one or more processors, cause the one or more processors to: generate cluster maps using clusters generated by performing clustering, wherein the generating the cluster maps performed based on a reference frame of a video generated based on a previous point in time of the video, generate a predicted frame by performing motion compensation based on the cluster maps, and generate a decoded frame by performing decoding based on the current frame and the predicted frame.