Methods and systems for creating a measurement model based only on measured training data are presented. The trained measurement model is then used to calculate overlay values directly from measured scatterometry data. The measurement models receive scatterometry signals directly as input and provide overlay values as output. In some embodiments, overlay error is determined from measurements of design rule structures. In some other embodiments, overlay error is determined from measurements of specialized target structures. In a further aspect, the measurement model is trained and employed to measure additional parameters of interest, in addition to overlay, based on the same or different metrology targets. In some embodiments, measurement data from multiple targets, measurement data collected by multiple metrologies, or both, is used for model building, training, and measurement. In some embodiments, an optimization algorithm automates the measurement model building and training process.