A method for curvilinear object segmentation includes receiving at least one input image comprising curvilinear features. The at least one input image is mapped to segmentation maps of the curvilinear features using a deep network having a representation module and a task module. The mapping includes transforming the input image in the representation module using learnable filters configured to balance recognition of curvilinear geometry with reduction of training error. The segmentation maps are produced using the transformed input image in the task module.