A sheet conveying device includes a conveying roller pair, a correction unit, and a control unit. The correction unit includes a correction roller pair and a roller holder. The control unit moves the roller holder from a reference position, and returns the roller holder to the reference position after the sheet is nipped by the correction roller pair, and after this sheet correction operation, the control unit releases the nip of the conveying roller pair. At least either a swing speed vh when returning the roller holder to the reference position, or a deflection amount of the sheet before the roller holder is returned to the reference position, is adjusted so that tension is not applied to the sheet between the conveying roller pair and the correction roller pair.