Real-time cloud detection, tracking, and trajectory forecasting using deep learning and sky imagery

Abstract

The intermittency of solar irradiance is caused by rapid cloud movement, and this poses a significant challenge for large-scale grid-integrated solar energy systems. This study presents a real-time deep learning-based framework for detecting, tracking, and forecasting the trajectories of clouds using ground-based Total Sky Imagery (TSI). A YOLOv8m model is used in cloud and sun detection as a robust approach, while BoT-SORT is used for real-time multi-object tracking. To predict cloud movement, a geometrical approach leveraging bounding box coordinates and velocity vectors is proposed, offering a more accurate method for short-term trajectory estimation. The system is tested on annotated datasets collected from a 1MW solar PV plant, demonstrating high detection accuracy and reliable real-time tracking performance. The results highlight the framework’s effectiveness as a foundational layer for solar forecasting applications and its potential for integration into smart grid management systems.