Some embodiments provide a method for training a machine-trained (MT) network that processes images using multiple network parameters. The method propagates a triplet of input images through the MT network to generate an output value for each of the input images. The triplet includes an anchor first image, a second image of a same category as the anchor image, and a third image of a different category as the anchor image. The method calculates a value of a loss function for the triplet that is based on a probabilistic classification of an output value for the anchor image compared to output values for the second and third images. The method uses the calculated loss function value to train the network parameters.