Provided are an apparatus and a method for manipulating depth of a stereoscopic image that enable a user to manipulate depth of a stereoscopic image easily and intuitively. The apparatus may include: a histogram analyzing unit acquiring a depth distribution chart by performing histogram analysis of a disparity map corresponding to an input image; a Gaussian mixture model (GMM) fitting unit acquiring multiple depth layers by performing GMM fitting of the depth distribution chart; a disparity adjusting unit adjusting at least one of a position and a volume of at least one of the multiple depth layers and calculating a disparity mapping function on which a result of the adjustment is reflected in response to a user request; and a disparity map recalculating unit calculating a new disparity map by optimizing the disparity mapping function.