The present disclosure provides a tracking system and method thereof. The tracking system comprises a trackable device with an appearance including a feature pattern and a tracking device. The tracking device comprises an optical sensor module configured to capture a first image which covers the trackable device. The tracking device further comprises a processor coupled to the optical sensor module. The processor is configured to retrieve a region of interest (ROI) of the first image based on the feature pattern, and locate a position of each of a plurality of feature blocks in the ROI, where each feature block contains a portion of the feature pattern. The processor further calculates a pose data of the trackable object according to the positions of the feature blocks.