Systems and methods for determining point-to-point distances from 3D image data. In some embodiments, two measure points, for example specified by a user, represent endpoints on an object of interest within an image frame. Assuming all points lying between these endpoints also belong to the object of interest, additional 3D data associated with points that lie along a measurement line defined by the measure points may be leveraged to provide a robust distance measurement. In some embodiments, total least squares fitting is performed, for example through Robust Principal Component Analysis (RPCA) to identify linear structures within the set of the 3D coordinates on the measurement line. In some exemplary embodiments, the minimum covariance determinant (MCD) estimator of the covariance matrix of the data is computed for a highly robust estimate of multivariate location and multivariate scatter.