A computer-implemented method is provided for keyphrase generation. The method includes pretraining, by a processor device, a policy neural network on training documents using a sequence-to-sequence model. The training documents are each associated with a list of keyphrases included therein. The method further includes training, by the processor device, the policy neural network using reinforcement learning with a summarization reward on present annotated keyphrases in an input training document and absent annotated keyphrase from the input training document that semantically describe a concept of the input training document. The method also includes predicting, by the processor device, new keyphrases using the trained policy neural network.