An online system matches a user across multiple online systems based on image data for the user (e.g., profile photo) regardless whether the image data is from the online system, a different but related online system or a third party system. For example, to match the user across a social networking system and INSTAGRAM™ system, the online system compares the similarity between images of the user from both systems in addition to similarity of textual information in the user profiles on both systems. The similarity of image data and the similarity of textual information associated with the user are used by the online system as indicators of matched user accounts belonging to the same user across both systems. The online system applies models trained using deep learning techniques to match a user across multiple online systems based on the image data and textual information associated with the user.