A method for determining a user plane security algorithm, a system, and an apparatus. The method may include: a second device selects a non-null user plane confidentiality protection algorithm based on a security capability of a first device and a security capability of the second device in a case in which user plane confidentiality protection between the second device and the first device is enabled and control plane confidentiality protection between the second device and the first device is not enabled. The second device sends a first message to the first device. The first message includes first algorithm indication information indicating the user plane confidentiality protection algorithm. Therefore, the first device can obtain the non-null user plane confidentiality protection algorithm. Embodiments can be adopted to determine an effective user plane confidentiality protection algorithm, for confidentiality protecting user plane data.