A method is provided for determining the relative overlay shift of stacked layers, said method comprising the steps of: a) providing a reference image including a reference pattern that comprises first and second pattern elements; b) providing a measurement image of a measurement pattern, which comprises a first pattern element formed by a first one of the layers and a second pattern element formed by a second one of the layers; c) weighting the reference or measurement image such that a weighted first image is generated, in which the first pattern element is emphasized relative to the second pattern element; d) determining the relative shift of the first pattern element on the basis of the weighted first image and of the measurement or reference image not weighted in step c); e) weighting the reference or measurement image such that a weighted second image is generated, in which the second pattern element is emphasized relative to the first pattern element; f) determining the relative shift of the second pattern element on the basis of the weighted second image and of the measurement or reference image not weighted in step e); g) determining the relative overlay shift on the basis of the relative shifts determined in steps d) and f).