In a system for moving, or dragging, a virtual reality environment, a user wearing a head mounted display (HMD) device may be at a first physical position in a physical space, corresponding to a first virtual position in the virtual environment. The user may select a second virtual position in the virtual environment by, for example, manipulation of a handheld electronic device operably coupled to the HMD. The system may construct a three dimensional complex proxy surface based on the first and second virtual positions, and may move the virtual elements of the virtual environment along the proxy surface. This movement of the virtual environment may be perceived by the user as a move from the first virtual position to the second virtual position, although the user may remain at the first physical position within the physical space.