The invention provides a prime number field elliptic curve cryptography system of a VLSI realization accelerator. The elliptic curve cryptography system comprises a modular arithmetic layer, a point arithmetic layer, an encryption master operation layer, an encryption mechanism layer, an external interface layer and a hardware resource layer. The accelerator is mounted on a bus as a slave device. A processor enables the accelerator to start working through configuring corresponding registers in the accelerator. According to the system, through rationally dividing hardware and software, an excellent performance to area ratio is obtained; and through adoption of hardware customization, prime number judgment, point addition and point multiplication operation efficiencies of a bottom layer are greatly improved.