A computer implemented method for determining shape from differential motion with unknown reflectance includes deriving a general relation that relates spatial and temporal image derivatives to bidirectional reflectance distribution function BRDF derivatives, responsive to 3D points and relative camera poses from images and feature tracks of an object in motion under colocated and unknown directional light conditions, employing a rank deficiency in image sequences from the deriving for shape determinations, under predetermined multiple camera and lighting conditions, to eliminate BDRF terms; and recovering a surface depth for determining a shape of the object.