Implementations are described herein for automatically annotating or curating digital images using various signals generated by individual users, in addition to or instead of content of the digital images themselves, thereby to enable the digital images to be retrieved from a searchable database based on their annotations. In particular, techniques are described herein for identifying events associated with a user, e.g., based on natural language input provided by a user, and automatically classifying/annotating images inferred to be related to those events.