Ch. Raga Madhuri; Dedeepya Manikonda; Madhurya Chinta

Evolutionary-Optimized Multimodal Driver Risk Assessment Using DE-Enhanced Cross-Attention Fusion of EEG and Telematics

PDF (877KB), PP.184-196

Views: 0 Downloads: 0

Author(s)

Ch. Raga Madhuri ^1,* Dedeepya Manikonda ² Madhurya Chinta ²

1. Department of Computer Science and Engineering, Siddhartha academy of Higher Education, Deemed to be University, Vijayawada 520 007. Andhra Pradesh, India

2. Department of Computer Science and Engineering, Student of Engineering, Velagapudi Ramakrishna Siddhartha Engineering College, Vijayawada -520007, Andhra Pradesh, India

* Corresponding author.

DOI: https://doi.org/10.5815/ijigsp.2026.04.10

Received: 20 Jan. 2026 / Revised: 16 Feb. 2026 / Accepted: 25 Mar. 2026 / Published: 8 Aug. 2026

Index Terms

EEG, CNN–LSTM, Telematics, XGBoost, PSO, Genetic Algorithm, Differential Evolution, Cross-Attention Fusion, Driver Safety, Multimodal Learning, Risk Assessment

Abstract

Road safety depends on both a driver’s emotional cognitive state and physical driving behavior, yet most existing systems rely on a single modality, limiting real-world reliability. This paper presents HECANet (Hybrid Evolutionary Cross-Attention Network), a multimodal framework that integrates EEG-based emotional cues and vehicle telematics behavior for robust driver risk assessment. EEG signals are modeled using a PSO-optimized CNN–LSTM to capture spatiotemporal emotional patterns, while telematics data are analyzed using a GA-optimized XGBoost model to identify safe, distracted, and aggressive driving behaviors. A Differential Evolution–optimized cross-attention fusion layer effectively aligns emotional and behavioral features, enabling interpretable emotion–behavior interaction modeling. The fused representation produces a driver safety score and risk probability, with K-Means clustering used to categorize drivers into Safe, Caution, and Risky groups. Experimental results achieve 94.7% accuracy and a 0.94 macro F1-score, demonstrating that joint emotion–behavior modeling significantly enhances driver risk prediction for intelligent transportation and fleet safety applications.

Cite This Paper

Ch Raga Madhuri, Dedeepya Manikonda, Madhurya Chinta, " Evolutionary-Optimized Multimodal Driver Risk Assessment Using DE-Enhanced Cross-Attention Fusion of EEG and Telematics", International Journal of Image, Graphics and Signal Processing(IJIGSP), Vol.18, No.4, pp. 184-196, 2026. DOI:10.5815/ijigsp.2026.04.10

Reference

[1]M. Jang and K. Oh, "Development of an integrated longitudinal control algorithm for autonomous mobility with EEG-based driver status classification and safety index," Electronics, vol. 13, no. 7, p. 1374, 2024, doi: 10.3390/electronics13071374.
[2]Chen, J., Lin, X., Ma, W., Wang, Y., & Tang, W. (2024). EEG-based emotion recognition for road accidents in a simulated driving environment. Biomedical signal processing and control, 87, 105411, 2024, doi: 10.1016/j.bspc.2023.105411.
[3]Liu, D., Yang, L., Ni, P., Wang, Q., Sun, H., Zhang, Q., & Tang, C. (2023, December). EEG-MLP: An all-MLP architecture for EEG emotion recognition. In 2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (pp. 2655-2662). IEEE, 2023, doi: 10.1109/BIBM58861.2023.10385434.
[4]P. Alrassy, A. W. Smyth, and J. Jang, "Driver behavior indices from large-scale fleet telematics data as surrogate safety measures," Accident Analysis & Prevention, vol. 179, p. 106879, 2023, doi: 10.1016/j.aap.2022.106879.
[5]Nikolaou, D., Ziakopoulos, A., Kontaxi, A., Theofilatos, A., & Yannis, G. (2025). Spatial analysis of telematics-based surrogate safety measures. Journal of Safety Research, 92, 98-108, 2025, doi: 10.1016/j.jsr.2024.09.012.
[6]J. Ju and H. Li, "A survey of EEG-based driver state and behavior detection for intelligent vehicles," IEEE Transactions on Biometrics, Behavior, and Identity Science, vol. 6, no. 3, pp. 420–434, 2024, doi: 10.1109/TBIOM.2024.3400866
[7]L. Wang, F. Song, T. H. Zhou, J. Hao, and K. H. Ryu, "EEG and ECG-based multi-sensor fusion computing for real-time fatigue driving recognition based on feedback mechanism," Sensors, vol. 23, no. 20, p. 8386, 2023, doi: 10.3390/s23208386.
[8]Angkan, P., Behinaein, B., Mahmud, Z., Bhatti, A., Rodenburg, D., Hungler, P., & Etemad, A. (2024). Multimodal brain–computer interface for in-vehicle driver cognitive load measurement: Dataset and baselines. IEEE Transactions on Intelligent Transportation Systems, 25(6), 5949-5964, 2024, doi: 10.1109/TITS.2023.3345846.
[9]Z. Lan, J. Zhao, P. Liu, C. Zhang, N. Lyu, and L. Guo, "Driving fatigue detection based on fusion of EEG and vehicle motion information," Biomedical Signal Processing and Control, vol. 92, p. 106031, 2024, doi: 10.1016/j.bspc.2024.106031.
[10]Y. Zhang, H. Guo, Y. Zhou, C. Xu, and Y. Liao, "Recognising drivers’ mental fatigue based on EEG multi-dimensional feature selection and fusion," Biomedical Signal Processing and Control, vol. 79, p. 104237, 2023, doi: 10.1016/j.bspc.2022.104237.
[11]M. R. Islam, S. Barua, M. U. Ahmed, S. Begum, P. Aricò, G. Borghini, and G. Di Flumeri, "A novel mutual information based feature set for drivers’ mental workload evaluation using machine learning," Brain Sciences, vol. 10, no. 8, p. 551, 2020, doi: 10.3390/brainsci10080551.
[12]D. Kulkarni and V. V. Dixit, "Hybrid classification model for emotion detection using EEG with improved feature set," Biomedical Signal Processing and Control, vol. 100, p. 106893, 2025, doi: 10.1016/j.bspc.2024.106893.
[13]S. Dai et al., "Contrastive learning of EEG representation of brain areas for emotion recognition," IEEE Transactions on Instrumentation and Measurement, 2025, doi: 10.1109/TIM.2025.3533618.
[14]A. R. Mussah and Y. Adu-Gyamfi, "Machine learning framework for real-time assessment of traffic safety utilizing connected vehicle data," Sustainability, vol. 14, no. 22, p. 15348, 2022, doi: 10.3390/su142215348.
[15]A. E. Abdelrahman, H. S. Hassanein, and N. Abu-Ali, "Robust data-driven framework for driver behavior profiling using supervised machine learning," IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 4, pp. 3336–3350, 2020, doi: 10.1109/TITS.2020.3035700.
[16]L. Shi, C. Qian, and F. Guo, "Real-time driving risk assessment using deep learning with XGBoost," Accident Analysis & Prevention, vol. 178, p. 106836, 2022, doi: 10.1016/j.aap.2022.106836.
[17]H. J. Li, X. G. Luo, Z. L. Zhang, W. Jiang, and S. W. Huang, "Driving risk prevention in usage-based insurance services based on interpretable machine learning and telematics data," Decision Support Systems, vol. 172, p. 113985, 2023, doi: 10.1016/j.dss.2023.113985.
[18]J. Hong, F. Liang, H. Zhang, Y. Chen, R. Li, K. Li, and J. Yang, "Data-driven multi-dimension driving safety evaluation for real-world electric vehicles," IEEE Transactions on Vehicular Technology, vol. 73, no. 7, pp. 9721–9733, 2024, doi: 10.1109/TVT.2024.3379677.
[19]J. Boylan, D. Meyer, and W. S. Chen, "A systematic review of the use of in-vehicle telematics in monitoring driving behaviours," Accident Analysis & Prevention, vol. 199, p. 107519, 2024, doi: 10.1016/j.aap.2024.107519.
[20]M. S. H. Nahid, T. Z. Tila, and T. S. Seecharan, "Detecting emotional arousal and aggressive driving using neural networks: A pilot study involving young drivers in Duluth," Sensors, vol. 24, no. 22, p. 7109, 2024, doi: 10.3390/s24227109.
[21]Pan, J., Fang, W., Zhang, Z., Chen, B., Zhang, Z., & Wang, S. (2023). Multimodal emotion recognition based on facial expressions, speech, and EEG. IEEE Open Journal of Engineering in Medicine and Biology, 5, 396-403, 2023, doi: 10.1109/OJEMB.2023.3240280.
[22]Bhatlawande, S., Shilaskar, S., Pramanik, S., & Sole, S. (2024). Multimodal emotion recognition based on the fusion of vision, EEG, ECG, and EMG signals. International journal of electrical and computer engineering systems, 15(1), 41-58, 2024, doi: 10.32985/ijeces.15.1.5.
[23]Wang, S., Qu, J., Zhang, Y., & Zhang, Y. (2023). Multimodal emotion recognition from EEG signals and facial expressions. IEEE Access, 11, 33061-33068, 2023, doi: 10.1109/ACCESS.2023.3263670.
[24]Koelstra, S., Muhl, C., Soleymani, M., Lee, J. S., Yazdani, A., Ebrahimi, T., ... & Patras, I. (2011). Deap: A database for emotion analysis; using physiological signals. IEEE transactions on affective computing, 3(1), 18-31, 2011, doi: https://doi.org/10.1109/T-AFFC.2011.15.
[25]Lin, C. T., Chang, C. J., Lin, B. S., Hung, S. H., Chao, C. F., & Wang, I. J. (2010). A real-time wireless brain–computer interface system for drowsiness detection. IEEE transactions on biomedical circuits and systems, 4(4), 214-222, 2010, doi: 10.1109/TBCAS.2010.2046415.
[26]Vasudevan, K., & Das, A. P. (2017, July). Driver drowsiness monitoring by learning vehicle telemetry data. In 2017 10th International Conference on Human System Interactions (HSI) (pp. 270-276). IEEE, 2017, pp. 270–276, doi: 10.1109/HSI.2017.8005054.
[27]Baltrušaitis, T., Ahuja, C., & Morency, L. P. (2018). Multimodal machine learning: A survey and taxonomy. IEEE transactions on pattern analysis and machine intelligence, 41(2), 423-443, 2018, doi: 10.1109/TPAMI.2018.2798607.
[28]Zhou, X., Lin, D., Jia, Z., Xiao, J., Liu, C., Zhai, L., & Liu, Y. (2023, July). An EEG channel selection framework for driver drowsiness detection via interpretability guidance. In 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 1-5). IEEE, 2023, doi: 10.1109/EMBC40787.2023.10341126.
[29]M. Winlaw, S. H. Steiner, R. J. MacKay, and A. R. Hilal, "Using telematics data to find risky driver behaviour," Accident Analysis & Prevention, vol. 131, pp. 131–136, 2019, doi: 10.1016/j.aap.2019.05.017.
[30]Dong, Y., Hu, Z., Uchimura, K., & Murayama, N. (2010). Driver inattention monitoring system for intelligent vehicles: A review. IEEE transactions on intelligent transportation systems, 12(2), 596-614, 2010, doi: 10.1109/TITS.2010.2092770.

International Journal of Image, Graphics and Signal Processing (IJIGSP)