A method of controlling polishing of a substrate is described. A controller stores a library having a plurality of reference spectra. The controller polishes a substrate and measures a sequence of spectra of light from the substrate during polishing. For each measured spectrum of the sequence of spectra, the controller finds a best matching reference spectrum from the plurality of reference spectra and generates a sequence of best matching reference spectra. The controller uses a cell counting technique for finding the best matching reference spectrum. The controller determines at least one of a polishing endpoint or an adjustment for a polishing rate based on the sequence of best matching reference spectra.