The invention discloses an unknown protocol recognizing method and system based on a bit stream. A frame header for an unknown protocol is recognized on the basis of the bit stream, the features of adata protocol bit stream are taken into full consideration, and an FST (Frequency Subsequence Trie) tree is constructed dynamically, and a frequent subsequence is calculated and found dynamically in order to determine a frame header part of the unknown protocol and effectively analyze a rule of the bit stream subsequence, so that an analysis result has higher adaptability, and defects in other data packet analysis methods are overcome. Moreover, a dynamical pruning principle is adopted, so that the FST tree does not grow excessively, and occupation of a memory is reduced. The unknown protocolrecognizing method and system have remarkable advantages on noise interference.