During a first horizontal blanking period, signal charges on a first row are transferred from a vertical transfer channel to a line memory. During a second horizontal blanking period, the transferred signal charges are partly transferred from the line memory to a horizontal transfer channel. During a third horizontal blanking period, the remaining signal charges on the first row are transferred from the line memory to the horizontal transfer channel. Thereafter, the signal charges on the second row next to the first row are transferred from the vertical transfer channel to a line memory that is vacant.