In one embodiment, a set of tags that has been generated by performing computer vision analysis of image content of a visual media item may be obtained, where each tag of the set of tags has a corresponding probability. In addition, a set of information that is independent from the image content of the visual media item may be obtained. The probability of at least a portion of the set of tags may be modified based, at least in part, upon the set of information.