A method and system for automatically performing a discovery of topics within temporal ordered text document collections are provided. The method includes generating a bag of words vector for each text document collection using a predefined dictionary. The method also includes iteratively calculating, based on the generated bag of words vectors, for each text document collection, a hidden topic vector representing topics of the respective text document collection using a calculated hidden state vector memorizing a hidden state of all previous text document collections.