GC aware containerized deployment to eliminate GC latencies

Abstract

Garbage collection in Java applications continues to be a major source of performance overhead, often causing disruptive pauses in latency sensitive workloads and reducing overall throughput. Although various techniques have been developed ranging from advanced garbage collection algorithms to finely tuned parameters, these strategies do not completely eliminate stop the world interruptions. This thesis presents a containerized deployment strategy aimed at achieving pause free execution for Java applications, leveraging Kubernetes orchestration and Kubernetes memory monitoring. The proposed methodology employs the Epsilon garbage collector, which inherently disables the garbage collection mechanism, thereby avoiding traditional garbage collection pause events. In place of conventional memory reclamation within a container, the system monitors heap usage and gracefully terminates containers approaching predefined memory thresholds, subsequently launching new containers to handle incoming traffic. This process ensures that application threads do not experience forced suspension for the purpose of heap compaction. Central to the design is the use of Kubernetes for container lifecycle management, complemented by Kubernetes metrics that detect memory pressure before critical levels are reached. When a container’s memory utilization exceeds a configurable limit, traffic is seamlessly redirected to a newly initiated container, allowing the retiring container to shut down without affecting ongoing requests or service availability. By shifting memory reclamation to container restarts, this approach prevents stop the world garbage collection pauses in a manner that reduces operational complexity. A comprehensive performance evaluation compares the container rotation strategy against the traditional Garbage collection algorithms. Experimental results indicate that the proposed solution consistently delivers lower latency profiles and higher throughput under varying workload conditions. Furthermore, offloading the memory reclamation task to container restarts lessens the need for intricate GC tuning, thus simplifying operational overhead. This thesis demonstrates a viable method of achieving truly pause free Java application execution, with implications for high availability systems and microservices that require minimal disruption from garbage collection.