The subject technology provides for receiving a new input stroke. The subject technology determines whether the new input stroke is associated with an existing line group based on a writing direction estimate of the existing line group. The subject technology merges the new input stroke with the existing line group in response to determining that the new input stroke is associated with the existing line group. The subject technology determines a local orientation of the existing line group including the new input stroke based on an estimate of a direction of writing and a scale of each stroke. The subject technology normalizes the existing line group including the new input stroke using the determined location orientation.