Real-time multi-lane speed measurement edge module with license plate recognition

Abstract

Overspeeding is a leading cause of road accidents. In 2024, Sri Lanka police reports indicated that over 90% of serious crashes were due to reckless driving. Automated speed enforcement systems have been adopted in many countries to address this issue, typically using radar or LiDAR to estimate vehicle speed and capturing images of vehicles that violate speed limits. Currently, in Sri Lanka, only handheld speed guns are used to detect speed violations. Installing fixed speed cameras in addition to speed guns can improve speed enforcement through fully automated operation and detecting speed violations of multiple vehicles simultaneously. Studies on fixed speed camera systems in other countries show that they lead to a notable reduction in severe and fatal crashes. However, implementing such systems in Sri Lanka presents several challenges. Most commercial solutions are designed for roadways where vehicles usually adhere to their lanes. On normal roads in Sri Lanka, this lane discipline is not strictly followed, and there might be multiple vehicles parallel on the same lane. Furthermore, smaller vehicles such as motorbikes are common in the country and are harder to detect. Existing systems are not customizable for these scenarios and are highly costly for a developing country. Hence, there is a need for a robust and low-cost static speed measurement device that can be easily deployed on Sri Lankan roads. This project presents a stationary edge module for multi-lane speed measurement and license plate recognition, customized for the above-mentioned challenges of Sri Lankan roads. The dual approach of capturing speed and license plate information allows automatic collection of speed violation data, enabling better enforcement of speed regulations and traffic management without increased manpower. Furthermore, the module performs real-time processing within 100 ms such that the speed and license plate number of passing vehicles can be instantly shown on an overhead display, serving as a visual deterrent for overspeeding. The device utilizes 77 GHz frequency-modulated continuous wave (FMCW) radar for speed and position estimation of multiple target vehicles. License plate recognition is done by passing camera-captured images into custom deep learning models for license plate detection followed by character recognition. A data fusion algorithm that involves time synchronization, spatial calibration, and point matching is used to accurately combine radar and image data. The hardware platform is built around the Zynq Ultrascale+ system-on-chip (SoC) with an integrated FPGA fabric, and it uses the Texas Instruments IWR1843 radar module. The FPGA is used for hardware acceleration of the deep learning models and data fusion unit, enabling real-time processing while keeping overall system power and cost low. To address the limitations of existing datasets when training the license plate recognition models, we created a dataset by capturing images from normal roads and expressways in Sri Lanka. The dataset contains 2970 multi-lane road images and 3425 unique license plate images. When trained on this dataset, the license plate detection model achieved 93.2% mAP and the character recognition model achieved 91.2% plate-level accuracy on test data. The models showed accurate performance in complex road scenarios and on motorbike license plates. Speed estimation was validated against vehicle speedometers, with a maximum error of 3 kmph. The system is theoretically capable of detecting speeds up to 250 kmph based on the radar specifications. Furthermore, in the field tests conducted in normal road, busy road, and expressway scenarios, the data fusion process matched the speed and license plate data with accuracies of 90.8%, 71.6%, and 94.5% respectively. Accuracy in busy road scenarios can be further improved through precise calibration. Using a camera operating at 10 FPS, the complete system achieved real-time speed estimation and license plate recognition with a maximum processing time of 70 ms per frame. These results demonstrate the effectiveness of the developed module as a reliable and affordable speed deterrent and enforcement system for Sri Lankan Road environments. Future work will focus on thorough verification of the speed estimation accuracy, integration with a cloud database for data storage and analysis and using IR-capable cameras to enable accurate nighttime license plate recognition.