Certain embodiments involve techniques for automatically identifying sounds in an audio recording that match a selected sound. An audio search and editing system receives the audio recording and preprocesses the audio recording into audio portions. The audio portions are provided as a query to the neural network that includes a trained embedding model used to analyze the audio portions in view of the selected sound to estimate feature vectors. The audio search and editing system compares the feature vectors for the audio portions against the feature vector for the selected sound and the feature vector for the negative samples to generate an audio score that is a numerical representation of the level of similarity between the audio portion and the selected sound and uses the audio scores to classify the audio portions into a first class of matching sounds and a second class of non-matching sounds.