A method of adjusting selected window size of an image object is applicable for tracking a target object in a video. The video includes a plurality of frames, and the target object has a display range changing with the playback of each frame. According to a variation trend of the display range of the target object, whether a variation times corresponding to the variation trend reaches a threshold value or not is recorded, and then the selected window size is reset, such that the target object is enclosed with a selected window having a size closer to the target object.