A tracking imaging control device includes a target color ratio calculation unit that calculates a ratio at which pixels with a certain range of hue occupy in the histogram as the target color ratio, with reference to the color of the target, and a tracking control unit that controls the pan and/or tilt operation of the camera to cause the camera to track the target, and the tracking control unit controls the pan and/or tilt operation of the camera on the basis of information on the position of the target detected by the first target detection unit when the target color ratio is equal to or lower than a threshold value, and controls the pan and/or tilt operation of the camera on the basis of information on the position of the target detected by the second target detection unit when the target color ratio exceeds the threshold value.