Benchmarking comparison between serverless vs. vm- architectures for AI workloads

Abstract

The rapid growth of Artificial Intelligence (AI) has led to an increased demand for cloud computing systems capable of handling diverse AI workloads. Serverless computing and VM-based cloud architectures are two dominant models for deploying AI applications, but choosing the appropriate model depends on performance factors such as latency, scalability, and cost-efficiency. This Systematic Literature Review (SLR) compares these two architectures, focusing on their effectiveness for AI tasks like real-time inference and AI model training. The review evaluates the performance metrics associated with each model, including cold-start latency, resource utilization, and scalability. Studies from 37 selected papers were analyzed using the PRISMA methodology, and key insights into the strengths and weaknesses of both models were extracted. Findings reveal that serverless systems excel in elastic scaling and cost-efficiency for bursty AI workloads but struggle with latency issues during cold starts. In contrast, VM-based systems offer predictable performance for long-duration tasks but often suffer from resource underutilization and higher costs. The review highlights the need for task-specific benchmarks and suggests the use of hybrid cloud models that combine the advantages of both approaches. Overall, this review contributes to the trade-offs between Serverless and VM-based cloud models for AI workloads and provides practical recommendations for future research and cloud deployment strategies.