A three-dimensional image processing apparatus and a method for controlling a location of a sweet spot for displaying a multi-view image are disclosed. A receiver receives a multi-view image including a plurality of view images. A controller detects a plurality of users from an image obtained by taking a watching zone, acquires user location information indicating locations of the plurality of detected users, calculates distance information indicating a distance between the detected users by using the acquired user location information, and controls a location of a sweet spot for viewing the plurality of view images on the basis of the calculated distance information and a length of a dead zone of the multi-view image.