Joint Optimization of Request Scheduling and Container Prewarming in Serverless Computing

Abstract

Serverless computing has emerged as a compelling paradigm for deploying applications and services due to its elastic scalability in response to changing demand. However, it often suffers from cold-start problems due to the overhead of initializing code and data dependencies. Thus, it’s necessary to keep a container warm after the container completes function processing and reuse it for subsequent requests. However, existing schedulers of Serverless platforms, which are usually load balancers, may not efficiently locate the reusable containers. Moreover, elastic scalers only scale passively based on predefined thresholds, making it challenging to handle burst requests. This paper introduces a scheduling algorithm called Consistent Hash-based Affinity Scheduling (CHAS), which aims to increase the chances of reusing warm containers by assigning functions to appropriate working nodes. We also propose a container prewarming strategy called LSTM-NB that uses a Long Short-Term Memory Network (LSTM) to predict the parameters of the negative binomial distribution (NB). This strategy performs joint learning of call time series of multiple functions and predicts future function calls to actively warm up or evict containers, thereby reducing cold start latency and excessive resource consumption. We build a serverless computing environment by using the SimPy discrete-event simulation framework to evaluate the proposed method. The results show that CHAS can reduce the cold start rate by an average of 10.5% compared to baseline approaches. Furthermore, an average reduction of 20.1% in the cold start rate can be achieved by prewarming with the LSTM-NB strategy.