A lithographic apparatus (LA) applies a pattern to a substrate (W). The lithographic apparatus includes a height sensor (LS), a substrate positioning subsystem, and a controller configured for causing the height sensor to measure the height (h) of the substrate surface at locations across the substrate. The measured heights are used to control the focusing of one or more patterns applied to the substrate. The height h is measured relative to a reference height (zref). The height sensor is operable to vary the reference height (zref), which allows a wider effective range of operation. Specifications for control of the substrate height during measurement can be relaxed. The reference height can be varied by moving one or more optical elements (566, 572, 576, 504 and/or 512) within the height sensor, or moving the height sensor. An embodiment without moving parts includes a multi-element photodetector (1212).