Hybrid Hierarchical Path Planning and Adaptive Tracking Control for Autonomous Vehicles via HDBHP Optimization and Reinforcement Learning

PDF (2000KB), PP.74-108

Views: 0 Downloads: 0

Author(s)

Lakshmi Narayana 1,2,* T. M. N. Vamsi 3

1. Department of Computer Science and Systems Engineering, AU-TDR HUB, Andhra University, Visakhapatnam, Andhra Pradesh 530003, India

2. Department of Artificial Intelligence and Data Science, Seshadri Rao Gudlavalleru Engineering College, Gudlavalleru, Vijayawada, Andhra Pradesh 521356, India

3. Department of Computer Science and Engineering, GITAM Deemed to be University, Visakhapatnam, Andhra Pradesh 530045, India

* Corresponding author.

DOI: https://doi.org/10.5815/ijisa.2026.05.05

Received: 21 Mar. 2026 / Revised: 15 May 2026 / Accepted: 17 Jul. 2026 / Published: 8 Oct. 2026

Index Terms

Autonomous Vehicles, Path Planning, Tracking Control, Hybrid Optimization, Reinforcement Learning

Abstract

The new hierarchical path planning and tracking control model for autonomous vehicles utilizes hybrid optimization techniques, an adaptive potential field approach, and reinforcement learning to achieve safe, efficient, and adaptive path planning of the vehicles. Initially, a dynamic environment is generated to process path planning and traffic control with obstacles. The global path is computed using the Hybrid Dung Beetle and Hippopotamus Optimization (HDBHP) algorithm for minimum path distance, minimum obstacles, and minimum energy. This path is smoothed using Robust Locally Weighted Regression with Curvature Smoothing (RLWR-CS) to fulfill the vehicle kinematic constraints and obtain smoothness. For real-time local modulation, the Tent Map-Artificial Potential Field (TM-APF) approach is used, which is sensitive to obstacles and uses a chaotic path to plan for a better response. Control is done through Goal-Conditional Q-learning (GCQL) and Prioritized Q-learning (PQL), where decision-making is made with specific goals and prioritized experience replay. An integrated reward function ensures that the path is accurate, safe, fast, and comfortable for the passengers and ensures that the vehicle adapts to the roads from the start to the destination. Simulation results demonstrate that the proposed approach achieves improved Cumulative Rewards of 171.704 at episode 968 and 196.141 at episode 9,251. Also, the presented approach has a lower execution time of 1.65 seconds and outperforms existing approaches such as Double Deep Q-Network (DDQN), Proximal Policy Optimization (PPO), Deep Q-Networks (DQN), and Soft Actor-Critic (SAC).

Cite This Paper

Lakshmi Narayana, T. M. N. Vamsi, "Hybrid Hierarchical Path Planning and Adaptive Tracking Control for Autonomous Vehicles via HDBHP Optimization and Reinforcement Learning", International Journal of Intelligent Systems and Applications (IJISA), Vol.18, No.5, pp.74-108, 2026. DOI: 10.5815/ijisa.2026.05.05

Reference

[1]Teng, Siyu, Xuemin Hu, Peng Deng, Bai Li, Yuchen Li, Yunfeng Ai, “Motion planning for autonomous driving: The state of the art and future perspectives”, IEEE Transactions on Intelligent Vehicles. Vol.8, No.6, pp.3692–3711, 2023. DOI: 10.1109/TIV.2023.3274536
[2]Yan, Yongjun, Lin Peng, Jinxiang Wang, Hui Zhang, Tong Shen, and Guodong Yin. "A hierarchical motion planning system for driving in changing environments: Framework, algorithms, and verifications." IEEE/ASME Transactions on Mechatronics 28, no. 3 (2022): 1303-1314. “A hierarchical motion planning system for driving in changing environments: Framework, algorithms, and verifications”, IEEE/ASME Trans Mechatron. Vol.28, No.3, pp.1303–1314, 2022. DOI: 10.1109/TMECH.2022.3219617
[3]He, Xiangkun, Haohan Yang, Zhongxu Hu, and Chen Lv. “Robust lane change decision making for autonomous vehicles: An observation adversarial reinforcement learning approach”, IEEE Transactions on Intelligent Vehicles. Vol.8, No.1, pp.184–193, 2022. DOI: 10.1109/TIV.2022.3165178
[4]Pérez-Gil, Óscar, Rafael Barea, Elena López-Guillén, Luis M. Bergasa, Carlos Gomez-Huelamo, Rodrigo Gutiérrez, and Alejandro Diaz-Diaz, “Deep reinforcement learning based control for autonomous vehicles in CARLA”, Multimedia Tools and Applications. Vol.81, No.3, pp.3553–3576, 2022. DOI: 10.1007/s11042-021-11437-3
[5]He, Yixu, Yang Liu, Lan Yang, and Xiaobo Qu, “Deep adaptive control: Deep reinforcement learning-based adaptive vehicle trajectory control algorithms for different risk levels”, IEEE Transactions on Intelligent Vehicles. Vol.9, No.1, pp.1654–1666, 2023. DOI: 10.1109/TIV.2023.3303408
[6]Zhang, Yuxiang, Xiaoling Liang, Dongyu Li, Shuzhi Sam Ge, Bingzhao Gao, Hong Chen, and Tong Heng Lee, “Adaptive safe reinforcement learning with full-state constraints and constrained adaptation for autonomous vehicles”, IEEE Transactions on Cybernetics. Vol.54, No.3, pp.1907–1920, 2023. DOI: 10.1109/TCYB.2023.3283771
[7]Tang, Xiaolin, Kai Yang, Hong Wang, Wenhao Yu, Xin Yang, Teng Liu, and Jun Li, “Driving environment uncertainty-aware motion planning for autonomous vehicles”, Chinese Journal of Mechanical Engineering, Vol.35, No.1, pp.120, 2022. DOI: 10.1186/s10033-022-00790-5
[8]Kim, Dongchan, Gihoon Kim, Hayoung Kim, and Kunsoo Huh, “A hierarchical motion planning framework for autonomous driving in structured highway environments”, IEEE Access. Vol.10, pp.20102–20117, 2022. DOI: 10.1109/ACCESS.2022.3152187
[9]Zhang, Yong, Kangting Liu, Feng Gao, and Fengkui Zhao, “Research on path planning and path tracking control of autonomous vehicles based on improved APF and SMC”, Sensors 23, Vol.23, No.18, pp.7918, 2023. DOI: 10.3390/s23187918
[10]Du, Guodong, Yuan Zou, Xudong Zhang, Zirui Li, and Qi Liu, “Hierarchical motion planning and tracking for autonomous vehicles using global heuristic based potential field and reinforcement learning based predictive control”, IEEE Transactions on Intelligent Transportation Systems 24, Vol.24, No.8, pp.8304–8323, 2023. DOI: 10.1109/TITS.2023.3266195
[11]Zhang, Xiao, Tong Zhu, Lei Du, Yueqi Hu, and Haoxue Liu, “Local path planning of autonomous vehicle based on an improved heuristic bi-RRT algorithm in dynamic obstacle avoidance environment”, Sensors 22, Vol.22, No.20, pp.7968, 2022. DOI: 10.3390/s22207968
[12]Zhang, Xinglong, Wenxin Zhang, Youqun Zhao, Hai Wang, Fen Lin, and Yingfeng Cai, “Personalized motion planning and tracking control for autonomous vehicles obstacle avoidance”, IEEE Transactions on Vehicular Technology, Vol.71, No.5, pp.4733–4747, 2022. DOI: 10.1109/TVT.2022.3152542
[13]Almazrouei, Khawla, Ibrahim Kamel, and Tamer Rabie, “Dynamic obstacle avoidance and path planning through reinforcement learning”, Applied Sciences, Vol.13, No.14, pp.8174, 2023. DOI: 10.3390/app13148174
[14]Huang, Zhiyu, Jingda Wu, and Chen Lv, “Efficient deep reinforcement learning with imitative expert priors for autonomous driving”, IEEE Transactions on Neural Networks and Learning Systems, Vol.34, No.10, pp.7391–7403, 2022. DOI: 10.1109/TNNLS.2022.3142822
[15]Bellotti, Francesco, Luca Lazzaroni, Alessio Capello, Marianna Cossu, Alessandro De Gloria, and Riccardo Berta, “Explaining a deep reinforcement learning (DRL)-based automated driving agent in highway simulations”, IEEE Access, Vol.11, pp.28522–28550, 2023. DOI: 10.1109/ACCESS.2023.3259544
[16]Li, Guofa, Siyan Lin, Shen Li, and Xingda Qu, “Learning automated driving in complex intersection scenarios based on camera sensors: A deep reinforcement learning approach”, IEEE Sensors Journal, Vol.22, No.5, pp.4687–4696, 2022. DOI: 10.1109/JSEN.2022.3146307
[17]Hu, Hui, Yuge Wang, Wenjie Tong, Jiao Zhao, and Yulei Gu, “Path planning for autonomous vehicles in unknown dynamic environment based on deep reinforcement learning”, Applied Sciences, Vol.13, No.18, pp.10056, 2023. DOI: 10.3390/app131810056
[18]N.S. Manikandan, Ganesan Kaliyaperumal, and Yong Wang, “Ad hoc-obstacle avoidance-based navigation system using deep reinforcement learning for self-driving vehicles”, IEEE Access, Vol.11, pp.92285–92297, 2023. DOI: 10.1109/ACCESS.2023.3297661
[19]Tang, Xiaolin, Guichuan Zhong, Shen Li, Kai Yang, Keqi Shu, Dongpu Cao, and Xianke Lin, “Uncertainty-aware decision-making for autonomous driving at uncontrolled intersections”, IEEE Transactions on Intelligent Transportation Systems, Vol.24, No.9, pp.9725–9735, 2023. DOI: 10.1109/TITS.2023.3283019
[20]Zhang, Sunan, Weichao Zhuang, Bingbing Li, Ke Li, Tianyu Xia, and Bo Hu, “Integration of planning and deep reinforcement learning in speed and lane change decision-making for highway autonomous driving”, IEEE Transactions on Transportation Electrification, 2024. DOI: 10.1109/TTE.2024.3393293
[21]Yang, Lei, Chao Lu, Guangming Xiong, Yang Xing, and Jianwei Gong, “A hybrid motion planning framework for autonomous driving in mixed traffic flow”, Green Energy and Intelligent Transportation, Vol.1, No.3, pp.100022, 2022. DOI: 10.1016/j.geits.2022.100022
[22]Antonio, Guillen-Perez, and Cano Maria-Dolores, “Multi-agent deep reinforcement learning to manage connected autonomous vehicles at tomorrow's intersections”, IEEE Transactions on Vehicular Technology,71, No.7, pp.7033–7043, 2022. DOI: 10.1109/TVT.2022.3169907
[23]Li, Guofa, Yifan Yang, Shen Li, Xingda Qu, Nengchao Lyu, and Shengbo Eben Li, “Decision making of autonomous vehicles in lane change scenarios: Deep reinforcement learning approaches with risk awareness”, Transportation Research Part C: Emerging Technologies, Vol.134, pp.103452, 2022. DOI: 10.1016/j.trc.2021.103452
[24]Albarella, Nicola, Dario Giuseppe Lui, Alberto Petrillo, and Stefania Santini, “A hybrid deep reinforcement learning and optimal control architecture for autonomous highway driving”, Energies. Vol.16, No.8, pp.3490, 2023. DOI: 10.3390/en16083490
[25]Dang, Fengying, Dong Chen, Jun Chen, and Zhaojian Li, “Event-triggered model predictive control with deep reinforcement learning for autonomous driving”, IEEE Transactions on Intelligent Vehicles, Vol.9, No.1, pp.459–468, 2023. DOI: 10.1109/TIV.2023.3329785
[26]Cao, Zhong, Diange Yang, Kun Jiang, Shaobing Xu, Sijia Wang, Minghan Zhu, and Zhongyang Xiao, “A geometry-driven car-following distance estimation algorithm robust to road slopes”, Transportation Research Part C: Emerging Technologies, Vol.102, pp.274–288, 2019. DOI: 10.1016/j.trc.2019.03.009
[27]Zhang, Bingzhan, Zhiyuan Li, Yaoyao Ni, and Yujie Li, “Research on path planning and tracking control of automatic parking system”, World Electric Vehicle Journal, Vol.13, No.1, pp.14, 2022. DOI: 10.3390/wevj13010014
[28]Xue, Jiankai, and Bo Shen, “Dung beetle optimizer: A new meta-heuristic algorithm for global optimization”, The Journal of Supercomputing, Vol.79, No.7, pp.7305–7336, 2023. DOI: 10.1007/s11227-022-04959-6
[29]Amiri, Mohammad Hussein, Nastaran Mehrabi Hashjin, Mohsen Montazeri, Seyedali Mirjalili, and Nima Khodadadi, “Hippopotamus optimization algorithm: A novel nature-inspired optimization algorithm”, Scientific Reports, Vol.14, No.1, pp.5032, 2024. DOI: 10.1038/s41598-024-54910-3
[30]Ma, Xiaolu, Rui Gong, Yibo Tan, Hong Mei, and Chengcheng Li, “Path planning of mobile robot based on improved PRM based on cubic spline”, Wireless Communications and Mobile Computing, Vol.2022, No.1, pp.1632698, 2022. DOI: 10.1155/2022/1632698
[31]Gabr, Mohamed, Hana Younis, Marwa Ibrahim, Sara Alajmy, and Wassim Alexan, “Visual data enciphering via DNA encoding, S-box, and tent mapping”, In 2022 IEEE 5th International Conference on Image Processing Applications and Systems (IPAS), pp.1–6, 2022. DOI: 10.1109/IPAS55744.2022.10052832
[32]Xie, Songtao, Junyan Hu, Parijat Bhowmick, Zhengtao Ding, and Farshad Arvin, “Distributed motion planning for safe autonomous vehicle overtaking via artificial potential field”, IEEE Transactions on Intelligent Transportation Systems, Vol.23, No.11, pp.21531–21547, 2022. DOI: 10.1109/TITS.2022.3189741
[33]B Eysenbach, Benjamin, Tianjun Zhang, Sergey Levine, and Russ R. Salakhutdinov, “Contrastive learning as goal-conditioned reinforcement learning”, Advances in Neural Information Processing Systems, Vol.35, pp.35603–35620, 2022. DOI: 10.48550/arXiv.2206.07568       
[34]Qu, Zhiyi, Xue Zhao, Huihui Xu, Hongying Tang, Jiang Wang, and Baoqing Li, “An improved Q-learning-based sensor-scheduling algorithm for multi-target tracking”, Sensors 22, Vol.22, No.18, pp.6972, 2022. DOI: 10.3390/s22186972