Quantum deep learning for encrypted malicious traffic detection a hybrid approach for secure network analysis

Abstract

Cybersecurity threats have become more advanced, making encrypted network traffic essential for security while also challenging to monitor. Transport Layer Security (TLS) is widely used to protect online communications by ensuring data integrity and confidentiality. However, a small number of malicious users exploit encryption for malicious activities, making it difficult to detect attacks using traditional Machine Learning (ML) and Deep Learning (DL) models. Which makes it very difficult to detect attacks using traditional Machine Learning (ML) and Deep Learning models. Majority of the Network Intrusion Detection Systems (NIDS) struggle with encrypted malicious network traffic. As decrypting the data for analysis would compromise the security, privacy and break the confidentiality of the servers, decrypting is not considered as good suggestion at most. Recent progress in Quantum Machine Learning (QML) provides a new path to detect anomalies in encrypted traffic without decrypting the traffic. Quantum Deep Learning can process complex network data more efficiently than classical methods. This research introduces a Hybrid Quantum Deep Learning framework, that combines Quantum Neural Networks (QNN), and Quantum Gated Recurrent Units (QGRU) to improve TLS malicious network traffic detection. This research evaluates the performance of QDL-based threat detection compared to traditional ML models. To be more specific in this VPN/Tor based TLS malicious network traffic detection will be explained through. It demonstrates that QDL models improve detection accuracy while maintaining efficient processing using real-world traffic datasets such as CIC-Darknet 2020.