A method and apparatus for communicating accumulated state information between internal and external tasks in a supervised learning system. A supervised learning system encodes state information for a hypothetical learning task on initialization. This hypothetical learning task state information indicates that no training instances have been received. During the supervised learning, training instances are presented to the supervised learner. The training instances are encoded with feature vector and target value information. For each task name paired with a non-default target value, the learner initializes a new learning task by copying the hypothetical learning task state representation for use as the state representation for the new learning task. Predictors are then produced for all learning tasks, except the hypothetical learning task. The new training instance is used to update all learning tasks as specified in the target vector. The new training instance is then used.to update the hypothetical learning task state representation as a negative example. Further training instances are handled similarly, new learning tasks are started based on the examination of the sparse target vector for task name, target value pairs which match received training instance target values and for which tasks have not yet been started.