An automatic tag identification system identifies tags for color themes, a color theme referring to a set of multiple colors that work well together to create (e.g., are visually appropriate for creating) a desired effect, and a tag referring to one or more words that describe a color theme. The automatic tag identification system receives an indication of the multiple colors (e.g., five colors) for a color theme. A first machine learning system uses the indicated multiple colors to generate a color theme embedding for the color theme, which is a vector encoding or embedding of the color theme. The second machine learning system uses the color theme embedding generated by the first machine learning system to generate one or more tags that label the color theme. These one or more tags can then be saved as associated with or corresponding to the multiple colors for the color theme.