An event-driven compute service upgrades an execution environment for registered functions of the compute service. Some compute nodes of the service have an updated version of the execution environment and some compute nodes have the old version of the execution environment. As a part of the upgrade process, some functions are executed in both environments. For example, for some invocations of a particular function, the function is executed in the updated execution environment while for other invocations of the same function, the function is executed in the old execution environment. Based on resulting status of the executions, the compute service determines that the particular function is compatible or incompatible with the updated version of the execution environment. The compute service performs subsequent executions of incompatible functions in the old version of the execution environment instead of in the updated version. A client may be notified of the incompatibility.