A video segmentation method for segmenting video clips according to content of the video clips is disclosed. The method comprises scanning pixels of video frames with a first horizontal scan line to determine if colors of the pixels fall within a predetermined color range; creating a color map utilizing pixels located on the first horizontal scan line from a plurality of successive video frames; labeling the current video segment as a candidate video segment if the color map indicates the presence of a stable region of pixels falling within the predetermined color range for a predetermined number of successive video frames; and performing histogram color comparisons on the stable regions for detecting shot transitions. Audio signals of the video clips may also be analyzed to further verify the candidate video segments.