Nizirwan Anwar; Titik Khawa Abdul Rahman; Aedah Abd Rahman; Swa Lee Lee; Muhammad Faisal; Dewanto Rosian Adhy; Tomy Ronaldi; Arief Ichwani

Benchmarking Supervised and Unsupervised Paradigms for Anomaly Detection in Autonomous Vehicle Telemetry: The Pitfalls of Internal Clustering Metrics

PDF (1181KB), PP.89-107

Views: 0 Downloads: 0

Author(s)

Nizirwan Anwar ^1,* Titik Khawa Abdul Rahman ² Aedah Abd Rahman ² Swa Lee Lee ³ Muhammad Faisal ⁴ Dewanto Rosian Adhy ⁵ Tomy Ronaldi ⁶ Arief Ichwani ¹

1. Department of Informatics Engineering, University of Esa Unggul, Jakarta, Indonesia

2. School Sains of Technology (SST), Asia e University, Malaysia

3. School Business Administration, Asia e University, Malaysia

4. Department of Informatics Engineering, University of Muhammadiyah Makassar, Indonesia

5. Department of Informatics Engineering, University of Mayasari Bakti, Tasikmalaya, Indonesia

6. Department of Defense Science, Republic of Indonesia Defense University Bogor, Indonesia

* Corresponding author.

DOI: https://doi.org/10.5815/ijwmt.2026.05.06

Received: 6 Jul. 2026 / Revised: 7 Aug. 2026 / Accepted: 10 Sep. 2026 / Published: 8 Oct. 2026

Index Terms

Autonomous vehicle, anomaly detection, machine learning, supervised classification, unsupervised clustering, ensemble methods, kinematic telemetry

Abstract

Anomaly detection in autonomous vehicle (AV) telemetry is a safety-critical task requiring accurate identification of abnormal vehicle behavior from multivariate kinematic sensor streams. This study presents a comprehensive comparative benchmark of twelve supervised classification algorithms and thirteen unsupervised clustering configurations applied to a real-world AV telemetry dataset comprising 112,028 observations and 23 engineered kinematic features after removal of seven constant-value columns. The dataset exhibits an approximately balanced class distribution (Normal: 56,380; Anomaly: 55,648). All classifiers were trained on an 80/20 stratified split and evaluated on Accuracy, Precision, Recall, F1-Score, and AUC-ROC. Clustering algorithms were evaluated on 15,000 randomly sampled observations using Silhouette Score, Davies-Bouldin Index (DBI), Calinski-Harabasz Index (CHI), Adjusted Rand Index (ARI), and Normalized Mutual Information (NMI). Results demonstrate that tree-based ensemble classifiers achieve performance approaching theoretical upper bounds — Extra Trees, Random Forest, Bagging, and Decision Tree (each F1 = 0.9998) — driven by the dominant discriminative power of speed, speed_time_ratio, rolling_distance_mean, and distance, which collectively account for 59.7% of Random Forest feature importance. In contrast, all unsupervised clustering algorithms fail to semantically recover ground-truth anomaly labels: the best-Silhouette configurations (Birch k=3, Silhouette = 0.9529) achieve ARI ≈ 0.000, while the best-ARI configuration (K-Means k=2, ARI = 0.440) recovers only partial anomaly structure. Three robustness experiments qualify these figures: 5-fold cross-validation confirms stability under random partitioning (F1 = 0.9998 ± 0.0001 for the top ensembles), while strictly temporal and unseen-scenario partitioning reduce peak F1 to 0.9750 and 0.9602 respectively, quantifying the contribution of window- and scenario-level information leakage to the near-saturated hold-out scores; a ten-sample sensitivity analysis shows the K-Means label alignment to be sample-dependent (ARI = 0.20 ± 0.21) whereas the degenerate high-Silhouette collapse is fully robust. Under the default configurations and static distribution assumption examined here, these findings indicate that supervised classification is the preferred paradigm for AV anomaly detection when labeled telemetry is available, and establish that Silhouette Score alone is a misleading model selection criterion in this context. All experiments were run with default hyperparameters in a documented software environment; robustness under hyperparameter tuning and distribution drift remains for future work.

Cite This Paper

Nizirwan Anwar, Titik Khawa Abdul Rahman, Aedah Abd Rahman, Swa Lee Lee, Muhammad Faisal, Dewanto Rosian Adhy, Tomy Ronaldi, Arief Ichwani, "Benchmarking Supervised and Unsupervised Paradigms for Anomaly Detection in Autonomous Vehicle Telemetry: The Pitfalls of Internal Clustering Metrics", International Journal of Wireless and Microwave Technologies(IJWMT), Vol.16, No.5, pp. 89-107, 2026. DOI:10.5815/ijwmt.2026.05.06

Reference

[1]S. Baccari, M. Hadded, H. Ghazzai, H. Touati, and M. Elhadef, “Anomaly Detection in Connected and Autonomous Vehicles: A Survey, Analysis, and Research Challenges,” IEEE Access, vol. 12, pp. 19250–19276, 2024, doi: 10.1109/ACCESS.2024.3361829.
[2]J. R. V. Solaas, E. Mariconti, and N. Tuptuk, “Systematic Literature Review: Anomaly Detection in Connected and Autonomous Vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 26, no. 1, pp. 43–58, Jan. 2025, doi: 10.1109/TITS.2024.3495031.
[3]S. Rajapaksha, H. Kalutarage, M. O. Al-Kadri, A. Petrovski, G. Madzudzo, and M. Cheah, “AI-Based Intrusion Detection Systems for In-Vehicle Networks: A Survey,” ACM Comput. Surv., vol. 55, no. 11, pp. 1–40, Nov. 2023, doi:10.1145/3570954.
[4]D. Bogdoll, M. Nitsche, and J. M. Zollner, “Anomaly Detection in Autonomous Driving: A Survey,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE, Jun. 2022, pp. 4487–4498. doi: 10.1109/CVPRW56347.2022.00495.
[5]Z. Zamanzadeh Darban, G. I. Webb, S. Pan, C. Aggarwal, and M. Salehi, “Deep Learning for Time Series Anomaly Detection: A Survey,” ACM Comput. Surv., vol. 57, no. 1, pp. 1–42, Jan. 2025, doi:10.1145/3691338.
[6]G. Pang, C. Shen, L. Cao, and A. Van Den Hengel, “Deep Learning for Anomaly Detection,” ACM Comput. Surv., vol. 54, no. 2, pp. 1–38, Mar. 2022, doi:10.1145/3439950.
[7]P. Dixit, P. Bhattacharya, S. Tanwar, and R. Gupta, “Anomaly detection in autonomous electric vehicles using AI techniques: A comprehensive survey,” Expert Syst., vol. 39, no. 5, Jun. 2022, doi: 10.1111/exsy.12754.
[8]R. Kaur, D. Gabrijelčič, and T. Klobučar, “Artificial intelligence for cybersecurity: Literature review and future research directions,” Information Fusion, vol. 97, p. 101804, Sep. 2023, doi: 10.1016/j.inffus.2023.101804.
[9]K. Wang, A. Zhang, H. Sun, and B. Wang, “Analysis of Recent Deep-Learning-Based Intrusion Detection Methods for In-Vehicle Network,” IEEE Transactions on Intelligent Transportation Systems, pp. 1–12, 2022, doi: 10.1109/TITS.2022.3222486.
[10]M. Althunayyan, A. Javed, and O. Rana, “A survey of learning-based intrusion detection systems for in-vehicle networks,” Computer Networks, vol. 277, p. 112031, Mar. 2026, doi: 10.1016/j.comnet.2026.112031.
[11]H. Min et al., “Toward interpretable anomaly detection for autonomous vehicles with denoising variational transformer,” Eng. Appl. Artif. Intell., vol. 129, p.107601, Mar. 2024, doi:10.1016/j.engappai.2023.107601.
[12]A. Sezgin, R. Keskin, and A. Boyacı, “Anomaly Detection in Unmanned Aerial Vehicle Telemetry Using Automated Machine Learning,” Fırat Üniversitesi Mühendislik Bilimleri Dergisi, vol. 37, no. 2, pp. 699–709, Sep. 2025, doi: 10.35234/fumbd.1668498.
[13]M. H. J. Naeimi, A. Norouzi, and A. Abdi, “GRAD: Real-Time Gated Recurrent Anomaly Detection in Autonomous Vehicle Sensors Using Reinforced EMA and Multi-Stage Sliding Window Techniques,” arXiv preprint arXiv:2510.23327, Oct. 2025. [Online]. Available: http://arxiv.org/abs/2510.23327
[14]S. Han, X. Hu, H. Huang, M. Jiang, and Y. Zhao, “ADBench: Anomaly Detection Benchmark,” inAdvances in Neural Information Processing Systems (NeurIPS 2022), Datasets and Benchmarks Track, 2022. doi: 10.48550/arXiv.2206.09426.
[15]Y. Fang et al., “Anomaly diagnosis of connected autonomous vehicles: A survey,” Information Fusion, vol. 105, p. 102223, May 2024, doi: 10.1016/j.inffus.2024.102223.
[16]H. Sträng and T. Dinh, “An upper bound on the silhouette evaluation metric for clustering,” Pattern Recognit., vol. 178, p. 113402, Oct. 2026, doi: 10.1016/j.patcog.2026.113402.
[17]Z. Li, Y. Zhu, and M. Van Leeuwen, “A Survey on Explainable Anomaly Detection,” ACM Trans. Knowl. Discov. Data, vol. 18, no. 1, pp. 1–54, Jan. 2024, doi:10.1145/3609333.
[18]G. Ciaburro, “Machine fault detection methods based on machine learning algorithms: A review,” Mathematical Biosciences and Engineering, vol. 19, no. 11, pp. 11453–11490, 2022, doi: 10.3934/mbe.2022534.
[19]L. Grinsztajn, E. Oyallon, and G. Varoquaux, “Why do tree-based models still outperform deep learning on tabular data?,” Jul. 2022, [Online]. Available: http://arxiv.org/abs/2207.08815
[20]V. Borisov, T. Leemann, K. Seßler, J. Haug, M. Pawelczyk, and G. Kasneci, “Deep Neural Networks and Tabular Data: A Survey,” IEEE Trans. Neural Netw. Learn. Syst., vol. 35, no. 6, pp. 7499–7519, Jun. 2024, doi: 10.1109/TNNLS.2022.3229161.
[21]M. Saied, S. Guirguis, and M. Madbouly, “A comparative analysis of using ensemble trees for botnet detection and classification in IoT,” Sci. Rep., vol. 13, no. 1, p. 21632, Dec. 2023, doi: 10.1038/s41598-023-48681-6.
[22]Y. Dong, K. Chen, Y. Peng, and Z. Ma, “Comparative Study on Supervised versus Semi-supervised Machine Learning for Anomaly Detection of In-vehicle CAN Network,” in 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), IEEE, Oct. 2022, pp. 2914–2919. doi: 10.1109/ITSC55140.2022.9922235.
[23]Y. Dong, L. Zhang, H. Farah, A. Zgonnikov, and B. van Arem, “Data-Driven Semi-Supervised Machine Learning with Safety Indicators for Abnormal Driving Behavior Detection,” Transportation Research Record: Journal of the Transportation Research Board, vol. 2679, no. 5, pp. 319–334, May 2025, doi: 10.1177/03611981241306752.
[24]A. Fernández, S. García, M. Galar, R. C. Prati, B. Krawczyk, and F. Herrera, Learning from Imbalanced Data Sets. Cham: Springer International Publishing,2018. doi:10.1007/978-3-319-98074-4.
[25]R. Sauber-Cole and T. M. Khoshgoftaar, “The use of generative adversarial networks to alleviate class imbalance in tabular data: a survey,” J. Big Data, vol. 9, no. 1, p. 98, Aug. 2022, doi: 10.1186/s40537-022-00648-6.
[26]A. L. Suárez-Cetrulo, D. Quintana, and A. Cervantes, “A survey on machine learning for recurring concept drifting data streams,” Expert Syst. Appl., vol. 213, p. 118934, Mar. 2023, doi: 10.1016/j.eswa.2022.118934.
[27]B. Bischl et al., “Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges,” WIREs Data Mining and Knowledge Discovery, vol. 13, no. 2, Mar. 2023, doi: 10.1002/widm.1484.
[28]T. J. Chengula, J. Mwakalonge, S. Siuhi, and G. Comert, “Improving Road Safety with Ensemble Learning: Detecting Driver Anomalies Using Vehicle Inbuilt Cameras,” 2023. doi: 10.2139/ssrn.4573044.
[29]V. P. Chellapandi, L. Yuan, C. G. Brinton, S. H. Żak, and Z. Wang, “Federated Learning for Connected and Automated Vehicles: A Survey of Existing Approaches and Challenges,” IEEE Transactions on Intelligent Vehicles, vol. 9, no. 1, pp. 119–137, Jan. 2024, doi: 10.1109/TIV.2023.3332675.
[30]E. Schubert and P. J. Rousseeuw, “Fast and eager k-medoids clustering: O(k) runtime improvement of the PAM, CLARA, and CLARANS algorithms,” Inf. Syst., vol. 101, p. 101804, Nov. 2021, doi: 10.1016/j.is.2021.101804.
[31]M. Landauer, S. Onder, F. Skopik, and M. Wurzenberger, “Deep learning for anomaly detection in log data: A survey,” Machine Learning with Applications, vol. 12, p. 100470, Jun. 2023, doi: 10.1016/j.mlwa.2023.100470.
[32]I. Makris et al., “A comprehensive survey of Federated Intrusion Detection Systems: Techniques, challenges and solutions,” Comput. Sci. Rev., vol. 56, p.100717, May 2025, doi:10.1016/j.cosrev.2024.100717.
[33]X. Ma, J. Wu, S. Xue, J. Yang, C. Zhou, Q. Z. Sheng, H. Xiong, and L. Akoglu, “A Comprehensive Survey on Graph Anomaly Detection with Deep Learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 12, pp. 12012–12038, Dec. 2023, doi: 10.1109/TKDE.2021.3118815.
[34]J. Xu, H. Wu, J. Wang, and M. Long, “Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy,” in Proc. International Conference on Learning Representations (ICLR), 2022. [Online]. Available: https://openreview.net/forum?id=LzQQ89U1qm_
[35]S. Tuli, G. Casale, and N. R. Jennings, “TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data,” Proceedings of the VLDB Endowment, vol. 15, no. 6, pp. 1201–1214, 2022, doi: 10.14778/3514061.3514067.
[36]H. Pallapothu, G. Krishna, D. Sunil, T. Yadav, and S. Nagasundari, “Autonomous Vehicle dataset for Anomaly detection,” IEEE Dataport, Jan. 2025, doi: 10.21227/0q64-5f96.

International Journal of Wireless and Microwave Technologies (IJWMT)