Techniques for controlling a vehicle on and off a route structure in an environment are discussed herein. A vehicle computing system controls the vehicle along a route based on a route-based reference system. The vehicle computing system may determine to operate off the route, such as to operate in reverse, park, etc. The vehicle computing system may modify vehicle operations to an inertial-based reference system to navigate to a location off the route. The vehicle computing system may determine a vehicle trajectory to the location off the route based on a reference trajectory between a location on the route and the location off the route and a corridor associated therewith. The vehicle computing system may transition between the route-based reference system and the inertial-based reference system, based on a determination to operate on or off the route.