Disclosed is an apparatus for loading and unloading cargo at a sea-side of a ship mooring at a quay wall, thereby enabling cargo to be loaded or unloaded simultaneously at both sides of the container ship by both cargo-handling apparatuses at the sea-side and the land-side. The apparatus comprises: a barge having a mooring side corresponding to one side of the ship; a barge anchoring unit for anchoring and releasing the barge at and from a predetermined place; at least one container crane disposed on an upper surface of the barge and being movable in a longitudinal direction of the barge; and at least one drawbridge for connecting an end of the mooring side of the barge to the quay wall. The apparatus remarkably improves the cargo-handling capability of a harbor in comparison with that by the prior art. Further, the apparatus reduces the time for which a ship stays at a quay wall, thereby improving the availability of the quay wall and reducing the physical distribution cost.