Polycystic Ovary Syndrome (PCOS) is a prevalent endocrine condition affecting women of reproductive age, hallmarked by hormonal abnormalities, ovarian cysts, and metabolic issues. Early diagnosis is essential to prevent long-term effects such as infertility, diabetes, and cardiovascular issues. Conventional diagnostic approaches relying on manual interpretation of ultrasound images are time-consuming and error-prone. To overcome these limitations, we propose an automated diagnostic framework leveraging deep feature extraction and ensemble learning. Initially, ResNet50 is utilized as a convolutional feature extractor, and its extracted features are classified using ensemble of Random Forest (RF) and Gradient Boosting (GB) classifiers. Subsequently, we also employed the Swin Transformer which is a hierarchical vision transformer to extract deep features from ultrasound images, which were fed to Random Forest and Gradient Boosting classifiers. These features were handled separately from those of ResNet50, and no feature concatenation was done. Compared to the ResNet50-based ensemble model, which achieved a classification accuracy of 99.2%, the Swin Transformer–based ensemble model performed better by attaining the accuracy of 99.87%. Furthermore, Explainable AI approaches (Grad-CAM) were applied to both ResNet50-based model and Swin Transformer-based model to highlight key regions contributing to the predictions. This scalable and interpretable system offers encouraging potential for advancing PCOS detection and other medical imaging applications.