An apparatus for tracking a location of a wireless terminal based on an image is provided. The apparatus includes an image matching database to store images classified by regions, and a matcher to receive a query image captured by a camera module of the wireless terminal and image matching data that contains region information about where the query image is captured, to search the image matching database for a matching image that corresponds to the image matching data, to determine if the query image matches the matching image, and to send the wireless terminal location information that corresponds to the matching information if the query image matches the matching image.