A document conveying device of the present disclosure includes a feeding unit, a stopper, and a restriction surface. The feeding unit is swingably supported on a first swinging fulcrum and is disposed selectively at a paper feeding position or at a retracted position. The stopper is swingably supported on a second swinging fulcrum, and has a document restriction portion and an engagement piece. The restriction surface is provided on the main body side of the document conveying device, and extends downward from a position coinciding with a first trajectory toward an inner side of the first trajectory, while extending downward from a position coinciding with a second trajectory toward an inner side of the second trajectory. When the feeding unit is at the retracted position, the engagement piece is engaged with the restriction surface, and when the feeding unit is at the paper feeding position, the engagement therebetween is released.