Scale Up while Scaling Out Microservices in Video Analytics Pipelines

/in /by

Publication Date: 12/18/2023

Event: Performance Optimization and Auto-Tuning of Software on Multicore/Manycore Systems (POAT 2023), Singapore

Reference: pp 1-8, 2023

Authors: Priscilla Benedetti, Vrije Universiteit Brussel – Brussels, Belgium, University of Perugia – Perugia, Italy; Giuseppe Coviello, NEC Laboratories America, Inc.; Kunal Rao, NEC Laboratories America, Inc.; Srimat T. Chakradhar, NEC Laboratories America, Inc.

Abstract: Modern video analytics applications comprise multiple microservices chained together as pipelines and executed on container orchestration platforms like Kubernetes. Kubernetes automatically handles the scaling of these microservices for efficient application execution. There are two popular choices for scaling microservices in Kubernetes i.e. scaling Out using Horizontal Pod Autoscaler (HPA) and scaling Up using Vertical Pod Autoscaler (VPA). Both these have been studied independently, but there isn’t much prior work studying the joint scaling of these two. This paper investigates joint scaling, i.e., scaling up while scaling out (HPA) is in action. In particular, we focus on scaling up CPU resources allocated to the application microservices. We show that allocating fixed resources does not work well for different workloads for video analytics pipelines. We also show that Kubernetes’ VPA in conjunction with HPA does not work well for varying application workloads. As a remedy to this problem, in this paper, we propose DataX AutoScaleUp, which performs efficiently scaling up of CPU resources allocated to microservices in video analytics pipelines while Kubernetes’ HPA is operational. DataX AutoScaleUp uses novel techniques to adjust the allocated computing resources to different microservices in video analytics pipelines to improve overall application performance. Through real-world video analytics applications like Face Recognition and Human Attributes, we show that DataX AutoScaleUp can achieve up to 1.45X improvement in application processing rate when compared to alternative approaches with fixed CPU allocation and dynamic CPU allocation using VPA.

Publication Link: https://ieeexplore.ieee.org/document/10387844