Systems, computer-implemented methods, and computer program products to facilitate reducing error propagation when combining generative aligners and transition-based aligners are provided. According to an embodiment, a system can comprise a processor that executed components stored in memory. The computer executable components comprise a generative alignment component, an error propagation component, a discriminative parser, and a stochastic oracle policy component. The error propagation component can compute a posterior distribution over one or more hard alignments of parts given a pair of the generative alignment component. The discriminative parser can be trained via the stochastic oracle policy component to reduce error propagation when combining the generative alignment component with the discriminative parser.