A method of generating neural network training image annotations includes training a first neural network to identify and segment hands in images using a first set of 2D images with hand portions segmented in each image; substantially simultaneously capturing both a second set of 2D images, and a third set of images including depth images, depicting hands holding a particular type of object; correlating each of the second set of images with corresponding images of the third set to identify and segment foregrounds from backgrounds in the second set of images; applying the trained first neural network to the identified foregrounds to identify hand portions of the foregrounds and segment object portions from identified hand portions; and training a second neural network, using the segmented object portions of the second set of images as training data, to identify the particular type of object in new images.