Techniques are disclosed for programmatically allocating memory among competing services in a distributed computing environment. Characteristics of web request streams and formulas for cache hit rates and client response times are used to create an objective function for memory allocation, such that maximum benefit can be realized from the memory allocations. When a particular service is allocated more memory, it can store more of its objects in cache, which improves client response time. Optionally, information from service level agreements may be used as input to the memory allocation computations.