Virtual and augmented reality sports training environments are disclosed. A user interacts with virtual players in a simulated environment of a virtual reality sporting event. In some embodiments, the user's actions and decisions are monitored by the simulated environment. The environment evaluates the user's performance, and provides performance feedback based on the user's decisions and timing. In one application, real life pitchers and their pitching data are tracked and replicated in a simulated pitching environment. A team of users may practice against a simulation of a pitcher they are about to compete against, swinging at pitches that they would see in a real game. Such environments may maximize effective practice time for users, and help develop better players with improved decision-making skills. Some embodiments include various pitch recognition training, which may be against replicated real-life pitchers.