Disclosed herein are a stereo camera system and a method for controlling convergence, including: a camera unit photographing both-eyes images; a filter unit filtering signal values of pixels for each line for any one of the both-eyes images along a line direction to detect a reference line of any one image; a line memory unit storing data for the reference line and a reference line of the other one image corresponding to the reference line; and a convergence control unit calculating the image control amount so as to align convergences of the both-eyes images by performing a comparison operation on the data for the reference lines and generating an optimal synthesis image of the both-eyes images by applying the image control amount.