A system and method that determines a seam between pairs of adjacent images for panoramic image stitching is disclosed. The method includes receiving a sequence of images, determining a pair of adjacent images in the sequence of images, matching one or more objects corresponding to a same object identifier in the pair of adjacent images, determining a seam in an overlap region between the pair of adjacent images and determining a portion of pixels from each image of the pair of adjacent images to represent in a stitched panoramic image based on the seam.