A system and a method of performing an image matting on an image are provided. The method includes detecting, by an image processing system, one or more objects in the image; determining, by the image processing system, a confidence map associated with the image using one or more image segmentation techniques for each of the one or more objects; and generating, by the image processing system, a tri-map for each of the one or more objects in the image from the confidence map based on at least one of a size of each of the one or more objects in the image and a distance between a first pixel in the image and a second pixel in at least one of the one or more objects in the image, wherein the tri-map is used to perform the image matting.