System and method for detecting and classifying direct response advertisements. The system includes a unit for generating an advertisement candidate segment for an advertisement section detected from a broadcast stream a matching unit for determining whether the candidate segment matches each advertisement segment stored in a database (DB); a unit for, if the matching unit determines that a segment matching the candidate segment is not present, determining whether the candidate segment is a direct response advertisement; a registration unit for storing the candidate segment, determined to be a direct response advertisement, as an advertisement segment that is the direct response advertisement in the DB; and a direct response advertisement grouping unit for, if the matching unit determines that an advertisement segment matching the candidate segment is present, and the matching segment is a direct response advertisement, grouping the candidate segment with DB-stored advertisement segments that are direct response advertisements.