A cargo conveying device includes: a horizontal direction conveying section that conveys cargo in a horizontal direction, and that includes a transfer section at which cargo is transferred between the transfer section and a cargo handler; a height adjusting section that adjusts a vertical direction height of the transfer section; and a control section that, at a time at a time at which cargo is transferred between the transfer section and a cargo handler, operates the height adjusting section, and adjusts the vertical direction height of the transfer section.