An image data processing device and an image data processing system are provided. An image data processing device performs a process of inputting a plurality of image data items in which imaging ranges at least partially overlap, detecting a face of a person in an image indicated by the image data, and recognizing a personal attribute of the person on the basis of the detected face for each of the image data items, a process of generating map data, in which the recognized personal attribute has been recorded in association with a position of the person in the image indicated by the image data, for each of the image data items, a process of interpolating the personal attribute of the person who overlaps between a plurality of the map data items, and a process of combining the plurality of map data items after the interpolation to generate composite map data.