A depth-sensing camera may be used to locate objects on a planar surface. A planar fit calculation is used to model the surface. One problem with the best planar fit calculation is that it is not robust over time. Firstly, this is because the depth data has a non-linear curve near its extremities because of camera lens distortion. Secondly, as the depth-sensing camera warms up and its thermal activity changes, the depth data drifts and the initial plane fit is no longer valid. The position and the amount of curvature and drift may be different for cameras of the same make and model in some cases. In accordance with some embodiments, algorithms may be used to detect and correct for these types of errors to support more robust plane fitting in real time.