A method, an apparatus and a computer program product are provided, wherein the method comprises receiving a video comprising video frames as an input; generating set of object proposals from the video, the set of object proposals comprising positive object proposals and negative object proposals; generating object tracklets comprising regions appearing in consecutive frames of the video, said regions corresponding to object proposals with a high confidence; constructing a graph for the object proposals to rescore the object proposals in the generated object tracklets; and aggregating the rescored object proposals to produce an object detection.