Techniques for modifying a garment based on gestures are presented herein. An access module can access a first set of sensor data from a first sensor, and a second set of sensor data from a second sensor. A garment simulation module can generate a three-dimensional (3D) garment model of a garment available for sale draped on an avatar based on the first set of sensor data and the second set of sensor data. A display module can cause a presentation, on a display of a device, of the 3D garment model draped on the avatar. Additionally, the garment simulation module can determine a modification gesture associated with the 3D garment model draped on the avatar based on the first set of sensor data and the second set of sensor data. Furthermore, the garment simulation module can modify the 3D garment model based on the determined modification gesture.