Systems and methods for topic modeling are described. The systems and methods include encoding words of a document using an embedding matrix to obtain word embeddings for the document. The words of the document comprise a subset of words in a vocabulary, and the embedding matrix is trained as part of a topic attention network based on a plurality of topics. The systems and methods further include encoding a topic-word distribution matrix using the embedding matrix to obtain a topic embedding matrix. The topic-word distribution matrix represents relationships between the plurality of topics and the words of the vocabulary. The systems and methods further include computing a topic context matrix based on the topic embedding matrix and the word embeddings and identifying a topic for the document based on the topic context matrix.