Real time anomaly detection for containerized environments

Abstract

Anomalies in containerized environments pose a significant threat, given their poten- tial to escalate small failures into catastrophic outcomes. These anomalies, which can manifest in various forms, possess the capability to disrupt service level agreements and tarnish an organization’s reputation irreversibly. Thus, it becomes imperative to detect and address these anomalies promptly to minimize their adverse effects on busi- ness operations. In this study, we delve into the landscape of anomalies prevalent in containerized environments, focusing on understanding their diverse nature and the substantial impact they can have. In this comprehensive survey, our focus lies in the real-time detection of anoma- lies within Kubernetes environments, a critical aspect in ensuring the robustness and stability of modern containerized systems. To achieve this, we conducted an exten- sive literature review, delving into existing research and methodologies pertinent to anomaly detection within Kubernetes ecosystems. Furthermore, we conducted practical experiments by deploying applications on Azure Kubernetes Services, leveraging the inherent Kubernetes metrics API and Prometheus API to gather pertinent data. Employing sophisticated feature selection techniques, we curated datasets and trained a decision tree model capable of discerning anomalous patterns in real-time metrics streams. Through rigorous experimentation, we validated the efficacy of our approach, achieving a remarkable accuracy rate of 95% in anomaly prediction. This research underscores the significance of real-time anomaly detection in Kubernetes environments and offers tangible insights for enhancing the resilience of containerized infrastructures.