Reinforcement learning-based security vulnerability detection for microservices

Abstract

Microservices architecture (MSA) is the main architectural model for contemporary software systems due to its scalability, flexibility, and maintainability. Yet, there are major security concerns that emerge from the distributed and dynamic nature of microservices. Most of these risks and vulnerabilities are difficult to identify due to the complexity and the evolving nature of microservice based systems. Due to this reason, conventional security vulnerability detection methods designed for monolithic systems are inadequate and ineffective in microservice based systems. To address this shortcoming, this thesis investigates the use of Reinforcement Learning (RL) and offers a Proximal Policy Optimization (PPO)-based framework as an automated and adaptive tool for finding security vulnerabilities in microservices. “Online Boutique” application, which built on microservices architecture is utilized as a testbed for assessing the performance of the developed RL framework in finding security vulnerabilities. The PPO agent learns to interact with the system, simulate attacks, and find real-time security vulnerabilities. The study intends to mimic DoS attack situations targeting the Online Boutique application. The suggested approach is a scalable, consistent security testing tool able to evolve to identify developing threats and adapt to new security vulnerabilities. This study proposes a promising substitute for conventional manual testing for DoS attacks. The results illustrate the efficacy of the PPO framework in detecting vulnerabilities in the microservices environment, with implications for improving the security of microservices-based applications.