The invention discloses a friendly track detection system based on visual processing, which comprises a two-camera input end, a DSP/ARM two-core processor, a visual processing module and an output interface, wherein lenses of two cameras are kept on the same plane and can rotate in a vertical direction and a horizontal direction by using the same point as an axle center, the DSP/ARM two core processor is connected with the two-camera input end for acquiring images in real time, the visual processing module is connected with the DSP/ARM two-core processor for receiving image information in real time, and the visual processing module is connected with the output interface. The invention ensures that image processing on a DSP core and template matching on an ARM core can be processed in parallel by using a hardware platform of the DSP/ARM two-core processor, thereby improving the speed of searching friendly tracks and fully meeting the requirement for processing images in real time. The two used cameras simplify the three-dimensional scene recovery on a visual region, thereby being capable of reducing the calculation quantity of a three-dimensional scene and quickening the speed of calculating the distance and the direction of the friendly tracks.