Disclosed herein are system, method, and computer program product embodiments for analyzing contextual symbol information for document processing. In an embodiment, a language model system may generate a vector grid that incorporates contextual document information. The language model system may receive a document file and identify symbols of the document file to generate a symbol grid. The language model system may also identify position parameters corresponding to each of the symbols. The language model system may then analyze the symbols using an embedding function and neighboring symbols to determine contextual vector values corresponding to each of the symbols. The language model system may then generate a vector grid mapping the contextual vector values using the position parameters. The contextual information from the vector grid may provide increase document processing accuracy as well as faster processing convergence.