An object tracking method includes steps of obtaining multiple first classifications of pixels within a first focus frame in a first frame picture, wherein the first focus frame includes an object to be tracked and has a first rectangular frame in a second frame picture; performing a positioning process to obtain a second rectangular frame; and obtaining color features of pixels around the second rectangular frame sequentially and establishing multiple second classifications according to the color feature. The established second classifications are compared with the first classifications sequentially to obtain an approximation value, compared with a predetermined threshold. The second rectangular frame is progressively adjusted, so as to establish a second focus frame. By analyzing color features of the pixels of the object and with a classification manner, the efficacy of detecting a shape and size of the object so as to update information of the focus frame is achieved.