A method for detecting jittering in videos generated by a shaken camera to remove the jittering on the videos using neural networks is provided for fault tolerance and fluctuation robustness in extreme situations. The method includes steps of: a computing device, generating each of t-th masks corresponding to each of objects in a t-th image; generating each of t-th object motion vectors of each of object pixels, included in the t-th image by applying at least one 2-nd neural network operation to each of the t-th masks, each of t-th cropped images, each of (t-1)-th masks, and each of (t-1)-th cropped images; and generating each of t-th jittering vectors corresponding to each of reference pixels among pixels in the t-th image by referring to each of the t-th object motion vectors. Thus, the method is used for video stabilization, object tracking with high precision, behavior estimation, motion decomposition, etc.