In autonomous driving, Outdoor Scene (OS) understanding in unfavorable weather conditions is pivotal for enabling safe navigation. Conventional studies failed to capture the intricate pairwise object relationship within scenes in OS classification. To mitigate these challenges, this research introduces a dual framework bringing together a Panoptic Cycle-Generative Adversarial Network (PC-GAN) with a Transfer Learning-based Squeeze-and-Zero-Knowledge Spatial Excitation DenseNet (TL-SZKSE-DenseNet). The suggested PC-GAN achieves realistic cross-weather image translation with semantically aligned representations in clear and degraded scenes. The TL-SZKSE-DenseNet improves feature learning with the addition of a zero-knowledge spatial excitation mechanism that selectively enhances weather-independent and context-dependent features for enhanced discrimination in crowded scenes. This generative–discriminative hybrid method effectively overcomes structured and unstructured driving domain gaps, as demonstrated on benchmark datasets such as BDD and IndiaScene365. Primarily, the OS images are generalized to adverse conditions using Panoptic Cycle-Generative Adversarial Network (PC-GAN). After that, using saliency mapping, the most significant regions in the generalized output are highlighted. In the meantime, the significant features are extracted utilizing the Orangutan Optimization Algorithm (OOA). In addition, from the detected objects, the Object Frequency and Relationship (OFR) is analyzed. Later, using TL-SZKSE-DenseNet, the OS are classified based on the OFR and the selected features. The performance of our proposed approach was tested on our newly developed IndiaScene365 dataset as well as various benchmark datasets such as BDD100k. Our newly developed dataset, IndiaScene365, includes 3000 images that entail three different weather conditions, which are foggy weather, rainy weather, and poor light conditions along with clear day weather. The proposed approach records 94.87% accuracy, 93.24%, and 94.18% sensitivity on our dataset.