Methods, systems, and apparatus, including computer programs encoded on computer storage media, for implementing critical section subgraphs in a computational graph system. One of the methods includes executing a lock operation including providing, by a task server, a request to a value server to create a shared critical section object. If the task server determines that the shared critical section object was created by the value server, the task server executes one or more other operations of the critical section subgraph in serial. The task server executes an unlock operation including providing, by the task server, a request to the value server to delete the shared critical section object.