The embodiments of the present invention disclose a remote conference system and a method of performing the remote conference. The remote conference system includes a first image-displaying system, at a first conference location, configured to display images of a plurality of audiences; a detection system configured to detect motions or poses of at least part of heads of persons in the first conference location; a judgment system configured to judge whether a person who is talking is talking to a certain audience or some certain audiences of the plurality of audiences or is talking to all the audiences based on the motions or poses detected by the detection system; a second image-displaying system, at a second conference location, configured to display special images to the certain audience or some the audiences of the plurality of audiences when the judgment system has judged that the person who is talking is talking to the certain audience or some certain audiences of the plurality of audiences, such that the certain audience or some of the plurality of audiences is/are aware that the person who is talking is talking to him or them. Whereby, target audience(s) in the remote conference may know the person who is talking is talking to him or them.