Implementations are described herein for three-dimensional (“3D”) modeling of objects that target specific features of interest of the objects, and ignore other features of less interest. In various implementations, a plurality of two-dimensional (“2D”) images may be received from a 2D vision sensor. The plurality of 2D images may capture an object having multiple classes of features. Data corresponding to a first set of the multiple classes of features may be filtered from the plurality of 2D images to generate a plurality of filtered 2D images in which a second set of features of the multiple classes of features is captured. 2D-3D processing, such as structure from motion (“SFM”) processing, may be performed on the 2D filtered images to generate a 3D representation of the object that includes the second set of one or more features.