A system for coloring a monochromatic image of a scene, the system comprising a processing circuitry configured to: capture the monochromatic image of the scene utilizing a non-visible spectrum sensor, capable of capturing monochromatic images from a viewpoint having a given position and a given orientation; obtain one or more previously captured color images, covering at least part of the scene; determine a registration between the monochromatic image and the previously captured color images; render one or more rendered images, being the previously captured color images adjusted to the viewpoint by utilizing the registration; and generate a colored image of the scene by changing, for at least one given pixel of the monochromatic image, values of one or more chroma components of the given pixel in accordance with values of one or more chroma components of a corresponding pixel of the rendered images.