The present invention discloses a multicast key negotiation method suitable for group calling system and a system thereof. The method includes that: a user terminal (UT) negotiates about a unicast key with a base station (BS), derives an information encryption key and an integrity verifying key according to the unicast key, and registers a service group identifier that the UT belongs to at the BS; the BS notifies the UT the multicast key of the service group that the UT needs to apply, constructs a multicast key notification packet, and sends it to the UT; after receiving the multicast key notification packet sent by the BS, the UT obtains the multicast key of the service group that the UT needs to apply by decrypting a service group key application list, constructs a multicast key confirmation packet, and sends it to the BS; the BS confirms that the multicast key of the UT service group is built successfully according to the multicast key confirmation packet sent by the UT.