An information processing apparatus includes an input unit configured to input an annotation of at least one of an underline, a box, a character, a character string, a symbol and a symbol string to a displayed document, an annotation recognition unit configured to recognize a type of the annotation and a coverage of the annotation in the document, an intention estimation unit configured to estimate intention of a user based on the type of the annotation and information in the coverage, an action storage unit configured to store a plurality of actions, an action selection unit configured to select an action to be performed for the document from the action storage based on the intention estimated by the intention estimation unit, and an execution unit configured to execute the action selected by the action selection unit.