The new hierarchical path planning and tracking control model for autonomous vehicles utilizes hybrid optimization techniques, an adaptive potential field approach, and reinforcement learning to achieve safe, efficient, and adaptive path planning of the vehicles. Initially, a dynamic environment is generated to process path planning and traffic control with obstacles. The global path is computed using the Hybrid Dung Beetle and Hippopotamus Optimization (HDBHP) algorithm for minimum path distance, minimum obstacles, and minimum energy. This path is smoothed using Robust Locally Weighted Regression with Curvature Smoothing (RLWR-CS) to fulfill the vehicle kinematic constraints and obtain smoothness. For real-time local modulation, the Tent Map-Artificial Potential Field (TM-APF) approach is used, which is sensitive to obstacles and uses a chaotic path to plan for a better response. Control is done through Goal-Conditional Q-learning (GCQL) and Prioritized Q-learning (PQL), where decision-making is made with specific goals and prioritized experience replay. An integrated reward function ensures that the path is accurate, safe, fast, and comfortable for the passengers and ensures that the vehicle adapts to the roads from the start to the destination. Simulation results demonstrate that the proposed approach achieves improved Cumulative Rewards of 171.704 at episode 968 and 196.141 at episode 9,251. Also, the presented approach has a lower execution time of 1.65 seconds and outperforms existing approaches such as Double Deep Q-Network (DDQN), Proximal Policy Optimization (PPO), Deep Q-Networks (DQN), and Soft Actor-Critic (SAC).