In various examples, a technique for generating a simulated multi-speaker recording includes determining a first rate at which a first speech-based attribute occurs within a first portion of the simulated multi-speaker recording. The technique also includes computing a first difference between the first rate and a first target rate for the first speech-based attribute. The technique further includes determining, based at least on the first difference, a second rate at which the first speech-based attribute is to occur within a second portion of the simulated multi-speaker recording and generating the second portion of the simulated multi-speaker recording based at least on the second rate.