The present invention relates to a tracking method and a tracking device adopting multiple observation models with different life spans. The tracking method is suitable for tracking an object in a low frame rate video or with abrupt motion, and uses three observation models with different life spans to track and detect a specific subject in frame images of a video sequence. An observation model I performs online learning with one frame image prior to the current image, an observation model II performs online learning with five frames prior to the current image, and an observation model III is offline trained. The three observation models are combined by a cascade particle filter so that the specific subject in the low frame rate video or the object with abrupt motion can be tracked quickly and accurately.