Techniques for automatically scoring submissions to an online question-and-answer submission system are disclosed. According to one such technique, an initial set of user submissions are scored by human operators and/or automated algorithmic mechanisms. The submissions and their accompanying scores are provided as training data to an automated machine learning mechanism. The machine learning mechanism processes the training data and automatically detects patterns in the provided submissions. The machine learning mechanism automatically correlates these patterns with the scores assigned to the submissions that match those patterns. As a result, the machine learning mechanism is trained. Thereafter, the machine learning mechanism processes unscored submissions. The machine learning mechanism automatically identifies, from among the patterns that the machine learning mechanism has already detected, one or more patterns that these submissions match. The machine learning mechanism automatically scores these submissions based on the matching patterns and the scores that are associated with those patterns.