Adaptive server-driven video streaming for real-time suspicious activity detection

Abstract

Real-time video streaming and analytics are the solutions to many safety and management systems. Even though it is widely used for video analytics-oriented applications everywhere, it must first consider the importance bounded by inference accuracy and the usage of resources such as hardware or network. Especially internet-based video streaming applications (IOT) must be adjusted to make the best use of application-level quality and adjusted to the limited network infrastructures. With deep neural networks (DNNs), instead of outmoded video streaming techniques, new techniques for optimizing the visual quality, such as rapid compression or pruning of pixels in outside areas, can achieve high inference accuracy. Currently, most of the device-based surveillance video streaming hinges mainly on the camera, the video generation device, which manages which frames and pixels to transfer through the network. However, the video camera-driven technique has aided well for typical video streaming applications. Also, real-time analytics-oriented applications still suffer from the source-driven application-level approach, where the video generation devices can only estimate the quality, and it takes work to get the measurement of the user experience directly. Similarly, most streaming protocols are guided by video sensor equipment (sensors or cameras), where the device computing power is deficient compared with the computers / mobile phones we use in our day-to-day life. A novel approach of DNN-driven streaming (DDS), which can be used to transfer a limited quality camera clips stream to the distance server while saving a lot of network bandwidth, and the server process the advanced deep neural network model in parallelly to regulate with the higher quality of stream of video clips to increase the inference accuracy moreover the Quality of Experience (QoE) over the existing commercial vision analytics-based surveillance monitoring systems. In the vision-based surveillance monitoring industry, understanding the scene's context is one of the most important aspects of a new generation of video surveillance markets. Recognizing the scene context from just one glance at the image is relatively easy for a human. However, it is still hard for a machine to accomplish. With the help of deep-learning algorithms and scene-type labels as a guide, we can identify the scenes to create a balanced dataset for training with advanced instance identification models. In this research, I implemented an adaptive DNN-driven streaming application with a novel approach for real-time identifying suspicious activities and unique scene detection. The latter part of this research will propose a generalized form of video analytics pipeline (VAP) approach, which can be suited for any AI-driven domain scenario. The newer version of the server-driven video streaming project for suspicious activity detection with demonstrations can be set up on any machine with GPU. Also, this research provides a custom suspicious activity detection dataset (based on the CAVIAR dataset[1], 2004), which is publicly available at https://github.com/sampaththennakoon/caviar_data_set, and the upgraded version of the server driven video streaming for suspicious activity detection project is available for download at https://github.com/sampaththennakoon/dds-v2.