A device may receive a first image. The device may process the first image to identify an object in the first image and a location of the object within the first image. The device may extract a second image from the first image based on the location of the object within the first image. The device may process the second image to determine at least one of a coarse-grained viewpoint estimate or a fine-grained viewpoint estimate associated with the object. The device may determine an object viewpoint associated with the second vehicle based on the at least one of the coarse-grained viewpoint estimate or the fine-grained viewpoint estimate. The device may perform one or more actions based on the object viewpoint.