A method of handwriting recognition is provided that combines the efficiency of a point oriented system and the shape information of a segment oriented system in an HMM based handwriting recognition system. The partial segmentation hypotheses obtained using the point oriented features in a conventional Viterbi search are augmented with scores based on segmental shape measurements made on the hypothesized segments. In addition, a method for segmentation of a handwriting sample is accomplished by incorporating a feature to distinguish between continuous and not continuous strokes and influencing the indicated hypothesis score by adding a penalty to the contraindicated hypothesis score.