An image forming apparatus has an image bearing member; toner image forming unit for forming a toner image with non-magnetic toner on an image bearing member; intermediate transfer member contactable to the image bearing member and movable along an endless path; primary transfer unit for effecting primary image transfer of the toner image formed on the image bearing member onto the intermediate transfer member at a first transfer position; secondary transfer unit for effecting a secondary image transfer of the toner image from the intermediate transfer member on the transfer material at a second transfer position; and a residual toner charger for charging the residual toner remaining on the intermediate transfer member to a polarity opposite from a regular polarity thereof. After the secondary image transfer, the residual toner remaining on the intermediate transfer member is charged by the residual toner charger to the opposite polarity, and is then passed through the first transfer position during the primary image transfer, so that the residual toner is transferred back to the image bearing member substantially simultaneously with a next primary image transfer, thus removing the residual toner from the intermediate transfer member.