A machine translation system can improve results of machine translations by employing preferred translations, such as human translated phrases. In some implementations, the machine translation system can use the preferred translations as heavily weighted training data when building a machine translation engine. In some implementations, the machine translation system can use the preferred translations as an alternate to a result that would have otherwise been produced by a machine translation engine. While it is infeasible to obtain human translations for all translation phrases, preferred translations can be used for problem phrases for which machine translation engines often produce poor translations. The machine translation system can identify problem phrases by assigning a quality score to each translation in a set of translations. The machine translation system can identify, as the problem phrases, n-grams that appear with a frequency above a frequency threshold in translations with quality scores below a threshold.