The present invention involves a tolerance based motion controller. The controller is capable of processing a group of tolerance constraints. The tolerance constraints specify where and when each tolerance constraint is to be applied, along with the information specifying the desired trajectory of motion. There are also a group of velocity constraints, specifying the maximum allowable velocity at each point along the desired trajectory. This information along with sensor feedback is used to modify the velocity along the actual trajectory of motion. This results in the time required traverse the trajectory being as short as possible. Also, the actual trajectory of motion should never exceed the permissible deviation from the desired trajectory, as specified by the tolerance constraints, with the velocity always being bounded by the specified velocity constraint.