A system and method for displaying map data to a person using either a hand-held or head-worn display is described. In one preferred system, a head-worn display is used. The location of the person and the direction of the person's view is determined. A map image indicated by the location and direction is presented in the display. This map image is displayed in a three-dimensional perspective view, allowing the person to easily associate the objects in the map image with the real objects in the person's field of view. In addition, the user may indicate a desired destination and the three-dimensional perspective view will be changed according to the user's location relative to the desired destination.