公开(公告)号：US20250165711A1

公开(公告)日：2025-05-22

申请号：US18649251

申请日：2024-04-29

Applicant: Shopify Inc.

Inventor： David Libbey ,  Neil Leonard Padgett

IPC: G06F40/284

Abstract: One problem of a generative language model (e.g. a large language model) is the generation of syntactically-invalid or misinformed output. This may be mitigated by utilizing a grammar defining valid sequences of output. The grammar may constrain the token generation. A method may include obtaining values generated using the generative language model, where each value is indicative of a probability of a respective token being a next token in the token sequence. The method may further include obtaining a mask based on the token sequence already generated and the grammar. The method may further include applying the mask to the values. The mask may operate on each value that corresponds to a token not compliant with the grammar to reduce or zero the probability of the corresponding token being the next token. The next token is then determined based on the values after the mask is applied.