The invention relates to a new method and system for detection and removal of rain from video using temporal/spatiotemporal properties. Advantageously, the temporal/spatiotemporal properties are involved to separate the rain pixels from the non-rain pixels. It is thus possible by way of the present invention to involve less number of consecutive frames, reducing the buffer size and delay. It works only on the intensity plane which reduces the complexity and execution time significantly along with accurate rain detection. This new technique does not assume the shape, size and velocity of the raindrops which makes it robust to different rain conditions. This method reduces the buffer size which reduces the system cost, delay and power consumption while maintaining sufficient quality of rain detection.