A video analytic device performs a method for detecting people within frames of video based upon multiple colors within their clothing. The method includes: receiving a frame of video; and determining that a first color region within the frame matches a first color of interest for a clothing uniform, wherein the determining is based on a first set of color representation constraints. The method further includes determining that a second color region within the frame matches a second color of interest for the clothing uniform, wherein the determining is based on a second set of color representation constraints and the first and second colors of interest are different. In addition, the method includes applying a set of geometric constraints to the first and second color regions to determine a count of people within the frame wearing the clothing uniform.