In the heat-shrinkable polyester film of the present invention, the main shrinkage direction is the longitudinal direction. In addition, the hot-water heat shrinkage ratios in the longitudinal direction and the width direction when the film is treated in hot water at 90° C. for 10 seconds, the refractive indexes in the longitudinal direction and the width direction and the natural shrinkage ratio after aging at 40° C. and 65% RH for 700 hours or more are each controlled in predetermined ranges. The heat-shrinkable polyester film of the present invention is manufactured by stretching an unstretched film at a ratio of 2.5 times or more and 6.0 times or less in the width direction at a temperature of Tg+5° C. or more and Tg+40° C. or less, passing the film in an intermediate zone that does not execute an active heating operation, heat-treating the film at a temperature of 100° C. or more and 170° C. or less over a period of 1.0 second or more and 10.0 seconds or less, cooling the film to a surface temperature of 30° C. or more and 70° C. or less, stretching the film at a ratio of 2.0 times or more and 7 times or less in the longitudinal direction at a temperature of Tg+5° C. or more and Tg+80° C. or less, and subsequently cooling the film at a cooling rate of 30° C./second or more and 70° C./second or less to a surface temperature of the film of 45° C. or more and 75° C. or less.