A welding system includes one or more cameras and a controller coupled to the one or more cameras. The one or more cameras are configured to detect a plurality of sets of visual markers of a welding device, where each set is oriented in a respective marker direction. The controller is configured to determine one or more marker directions of one or more respective sets of visual markers based on a detected set of visual markers, to select one of the sets of visual markers as a tracked set of visual markers based at least in part on a determined marker direction of the tracked set of visual markers, to associate a rigid body model to the tracked set of visual markers, and to determine a position and an orientation of the welding device based on the associated rigid body model of the tracked set of visual markers.