Techniques are described herein for grouping of operations in local memory of a processing unit. The techniques involve adding a first operation for a first leaf operator of a query execution plan to a first pipelined group. The query execution plan includes a set of leaf operators and a set of non-leaf operators. Each leaf operator of the set of one or more leaf operators has a respective parent non-leaf operator and each non-leaf operator has one or more child operators from among the set of leaf operators or others of the set of non-leaf operators. The techniques further involve determining a memory requirement of executing the first operation for the first leaf operator and executing a second operation for the respective parent non-leaf operator of the first leaf operator. The output of the first operation is input to the second operation. The techniques further involve determining whether the memory requirement is satisfied by an amount of local memory. If it is determined that the memory requirement is satisfied by the amount of local memory the second operation for the respective parent non-leaf operator is added to the first pipelined group. The techniques further involve assigning the first pipelined group to a first thread and the first thread executing the first pipelined group. Executing the first pipelined group involves: storing first output of the first operation in the local memory of the first thread; using the first output as input for the second operation; storing second output of the second operation in the local memory; and moving second output from the local memory to a tier of memory different than the local memory relative to the first thread.