A method including receiving input comprising natural language texts; pre-training a First-Order Logic Network (FOLNet) neural network model on unlabeled texts included in the natural language texts, the FOLNet neural network model comprising of a plurality of layers; processing the input through the plurality of layers of the FOLNet neural network model; encoding a logical inductive bias using the FOLNet neural network model; outputting one or more tensors based on the logical inductive bias; and predicting an outcome using the one or more tensors.