The invention relates to an unshielded SERF (Spin Exchange Relaxation Free Regime) atomic magnetometer magnetic field cancellation coil non-orthogonal angle measuring device and a measuring method. The center of a three-axis magnetic field cancellation coil 1 is provided with a scalar magnetometer probe 4; an x coil is excited by a driving current source I; a y coil is excited by a driving current source II; a z coil is excited by a driving current source III; and output signals of the scalar magnetometer probe 4 are accessed to a phase-locked amplifier 5 through a precision square arithmetic circuit 6. Different from the existing measuring method of carrying out magnetic field modulation on the magnetometer, carrying out fitting processing on output data and acquiring the non-orthogonal angles of the X axis and the Y axis of the coil through deduction, the method of the invention can measure the non-orthogonal angle through simple calculation according to data directly after magnetic field modulation. As restrictions from conditions of the SERF magnetometer itself do not exist, a large amount of data measured for fitting are not needed, three non-orthogonal angles of the coils can be all meausred, and the problem of data calibration in the SERF magnetometer experiment can be solved.