Methods and systems for learnable defect detection for semiconductor applications are provided. One system includes a deep metric learning defect detection model configured for projecting a test image for a specimen and a corresponding reference image into latent space, determining a distance in the latent space between one or more different portions of the test image and corresponding portion(s) of the corresponding reference image, and detecting defects in the one or more different portions of the test image based on the determined distances. Another system includes a learnable low-rank reference image generator configured for removing noise from one or more test images for a specimen thereby generating one or more reference images corresponding to the one or more test images.