Cost optimized scheduling for microservices in kubernetes

Abstract

The usage of Container Orchestration Platform like Kubernetes for running Microservices applications is increasing nowadays. In a particular application, all Microservices do not have the same priority. Hence it is costly to allocate the same resources to both high and low-priority services. Spot instances are an attractive option for running low-priority services due to their significantly lower cost compared to On- Demand instances. Spot instances are available for use when cloud service providers have excess capacity and can be bid on at a much lower price than the On-Demand rate. But they can be revoked anytime by the Cloud provider which affects the availability of the services. This research aims to utilize Spot instances to run low-priority services with the intention of reducing the cloud cost while providing overall high availability to the application. A thorough literature review has been conducted on existing research that utilizes Spot instances to save cost while maintaining high availability. This study builds upon previous work and proposes a new approach to run low priority Microservices to save cost. A service called KubeEconomy has been proposed to monitor and manage Kubernetes worker nodes to efficiently schedule the Microservices. Three functionalities of the KubeEconomy service have been explained which contributes to the cost optimization. The KubeEconomy service utilizes cloud APIs and Kubernetes APIs to promptly scale and reschedule pods within different nodes. Two experiments were conducted to show the effectiveness of KubeEconomy service. In the first experiment, the KubeEconomy service was deployed on Azure cloud to manage a Kubernetes cluster. The experiment showed that the KubeEconomy service was able to dynamically provision and deprovision Spot instances based on the workload demand and Spot evictions, resulting in significant cost savings while maintaining high availability of the Microservices. In the second experiment, a simulation was conducted using the parameters gathered from the first experiment to calculate the cost savings of long running workloads. It is shown that it is possible to reduce the cloud cost up to 80% while maintaining 99% availability for the Microservices under optimal conditions.