A pipe handling system for handling the pipes from a cargo hull includes a gantry beam and a guide mast assembly having an essentially vertical guide mast. The guide mast assembly is moveably connected to the gantry beam, allowing movement of the guide mast assembly in a vertical direction between a lowered mast position and a lifted mast position relative to the gantry beam. A lifting part includes a device configured to engage a pipe. The lifting part is moveable by one or more hoists in a vertical direction between a lowered lifting part position for picking up the pipe and a lifted lifting part position. The lifted part engages on the guide mast assembly, such that the guide mast assembly causes vertical guidance of the lifting part when the guide mast assembly with the engaged lifting part is moved between the lowered mast position and the lifted mast position.