A robot system is configured to identify gestures performed by an end-user proximate to a work piece. The robot system then determines a set of modifications to be made to the work piece based on the gestures. A projector coupled to the robot system projects images onto the work piece that represent the modification to be made and/or a CAD model of the work piece. The robot system then performs the modifications.