International Journal of Image, Graphics and Signal Processing (IJIGSP)
IJIGSP Vol. 18, No. 2, 8 Apr. 2026
Cover page and Table of Contents: PDF (size: 1584KB)
Employing Counterfactual Methods to Interpret Convolutional Network Findings in X-Ray Image Detection
PDF (1584KB), PP.1-19
Views: 0 Downloads: 0
Author(s)
Maider Abad
1,*
Eusebio Garcia
1
Ferran Prados
1,2,3
Jordi Casas-Roma
1,4,5
Index Terms
X-ray Imaging, Counterfactual Explanation, RadImageNet, Image Classification, Explainable Artificial Intelligence (XAI)
Abstract
In the rapidly evolving landscape of medical diagnostics, efficient and accurate tools for disease identification are crucial. This study analyzes three convolutional neural network (CNN) architectures—IRV2, ResNet50, and DenseNet121—pre-trained on ImageNet and RadImageNet datasets for respiratory disease diagnosis using chest radiographs. We used over 10,000 chest X-ray images, including COVID-19, pneumonia, and control cases, to train and evaluate these models. RadImageNet-trained models, particularly ResNet50, achieved superior performance with 94.49% accuracy, 93.92% sensitivity, and 95.59% precision compared to ImageNet-trained counterparts, though the improvement was not statistically significant in most cases. To enhance interpretability, we developed a counterfactual-based method generating visual explanations of critical areas influencing diagnostic outcomes. This approach, not requiring access to training data or model internals, identifies image parts that could change the predicted diagnosis if altered. It aids in understanding model reasoning and can correct misclassifications, successfully reclassifying up to 40.91% of previously misclassified images through our masking method. By providing clear, independent visual explanations, our method aims to foster trust in AI-assisted diagnoses among medical professionals. While preliminary results are promising, further validation with medical experts will help confirm the clinical relevance of the highlighted regions. This will strengthen the transparency and interpretability of AI decision-making in healthcare. The visual nature of these explanations offers a valuable tool for interpreting complex medical image classification models and may enhance the synergy between AI systems and human expertise in diagnostic processes.
Cite This Paper
Maider Abad, Eusebio Garcia, Ferran Prados, Jordi Casas-Roma, "Employing Counterfactual Methods to Interpret Convolutional Network Findings in X-Ray Image Detection", International Journal of Image, Graphics and Signal Processing(IJIGSP), Vol.18, No.2, pp. 1-19, 2026. DOI:10.5815/ijigsp.2026.02.01
Reference
[1]Jia Song, Shaohua Gao, Yunqiang Zhu, and Chenyan Ma. A survey of remote sensing image classification based on CNNs. Big earth data, 3(3):232–254, 2019.
[2]Shervin Minaee, Rahele Kafieh, Milan Sonka, Shakib Yazdani, and Ghazaleh Jamalipour Soufi. Deep-COVID: Predicting COVID-19 from chest x-ray images using deep transfer learning. Medical Image Analysis, 65:101794, 07 2020.
[3]Simone Bianco, Remi Cadene, Luigi Celona, and Paolo Napoletano. Benchmark analysis of representative deep neural network architectures. IEEE access, 6:64270–64277, 2018.
[4]Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. IEEE, 2009.
[5]Roberto Poli, M.J. Healy, and Achilles Kameas. Theory and Applications of Ontology: Computer Applications. 01 2010.
[6]Xueyan Mei, Zelong Liu, Philip M Robson, Brett Marinelli, Mingqian Huang, Amish Doshi, Adam Jacobi, Chendi Cao, Katherine E Link, Thomas Yang, et al. Radimagenet: an open radiologic deep learning research dataset for effective transfer learning. Radiology: Artificial Intelligence, 4(5):e210315, 2022.
[7]Tom Vermeire, Dieter Brughmans, Sofie Goethals, Raphael Mazzine Barbossa De Oliveira, and David Martens. Explainable image classification with evidence counterfactual. Pattern Analysis and Applications, 25(2):315–335, 2022.
[8]Sainyam Galhotra, Romila Pradhan, and Babak Salimi. Explaining black-box algorithms using probabilistic contrastive counterfactuals. In Proceedings of the 2021 International Conference on Management of Data, pages 577– 590, 2021.
[9]Miguel Cardoso, Carlos Santiago, and Jacinto C Nascimento. Using counterfactual information for breast classification diagnosis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4996–5002, 2024.
[10]David Major, Dimitrios Lenis, Maria Wimmer, Gert Sluiter, Astrid Berg, and Katja Bu¨hler. Interpreting medical image classifiers by optimization based counterfactual impact analysis. In 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pages 1096–1100. IEEE, 2020.
[11]Alexandre Cadrin-Cheˆnevert. Moving from imagenet to radimagenet for improved transfer learning and generalizability. Radiology: Artificial Intelligence, 4(5):e220126, 2022.
[12]Ebrahim Ali Nehary, Sreeraman Rajan, and Carlos Rossa. Comparison of COVID-19 classification via imagenetbased and radimagenet-based transfer learning models with random frame selection. In 2023 IEEE Sensors Applications Symposium (SAS), pages 1–6. IEEE, 2023.
[13]Nihal Remzan, Karim Tahiry, and Abdelmajid Farchi. Advancing brain tumor classification accuracy through deep learning: harnessing radimagenet pre-trained convolutional neural networks, ensemble learning, and machine learning classifiers on mri brain images. Multimedia Tools and Applications, pages 1–29, 2024.
[14]Nihal Remzan, Karim Tahiry, and Abdelmajid Farchi. Efficient brain tumor classification on resource-constrained devices using stacking ensemble and radimagenet pretrained models. In 2023 6th International Conference on Advanced Communication Technologies and Networking (CommNet), pages 1–7. IEEE, 2023.
[15]Ruth Kassahun, Mario Molinara, Alessandro Bria, Claudio Marrocco, and Francesco Tortorella. Breast Mass Detection and Classification Using Transfer Learning on OPTIMAM Dataset Through RadImageNet Weights, pages 71–82. 01 2024.
[16]Arnaud Van Looveren and Janis Klaise. Interpretable counterfactual explanations guided by prototypes. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 650–665. Springer, 2021.
[17]Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee. Counterfactual visual explanations. In International Conference on Machine Learning, pages 2376–2384. PMLR, 2019.
[18]Vivek Narayanaswamy, Jayaraman J Thiagarajan, and Andreas Spanias. Using deep image priors to generate counterfactual explanations. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2770–2774. IEEE, 2021.
[19]Sumedha Singla, Motahhare Eslami, Brian Pollack, Stephen Wallace, and Kayhan Batmanghelich. Explaining the black-box smoothly—a counterfactual approach. Medical Image Analysis, 84:102721, 12 2022.
[20]Jayaraman J. Thiagarajan, Kowshik Thopalli, Deepta Rajan, and Pavan Turaga. Training calibration-based counterfactual explainers for deep learning models in medical image analysis. Scientific Reports, 12, 01 2022.
[21]1Ke Wang, Zicong Chen, Mingjia Zhu, Zhetao Li, Jian Weng, and Tianlong Gu. Score-based counterfactual generation for interpretable medical image classification and lesion localization. IEEE transactions on medical imaging, PP, 03 2024.
[22]Tawsifur Rahman. COVID-19 radiography database. Data retrieved from Kaggle, https://www.kaggle.com/datasets/tawsifurrahman/covid19-radiography-database, Mar 2022.
[23]Daniel Kermany. Labeled optical coherence tomography (OCT) and chest X-Ray images for classification. https://data.mendeley.com/datasets/rscbjbr9sj/2, January 2018.
[24]Maider Abad, Jordi Casas-Roma, and Ferran Prados. Generalizable disease detection using model ensemble on chest x-ray images. Scientific Reports, 14, 03 2024.
[25]S Tabik, A Gomez-Rios, J L Martin-Rodriguez, I Sevillano-Garcia, M Rey-Area, D Charte, E Guirado, J L Suarez, J Luengo, M A Valero-Gonzalez, P Garcia-Villanova, E Olmedo-Sanchez, and F Herrera. COVIDGR dataset and COVID-SDNet methodology for predicting COVID-19 based on chest X-Ray images. IEEE J Biomed Health Inform, 24(12):3595–3605, December 2020.
[26]Unais Sait, KG Lal, S Prajapati, Rahul Bhaumik, Tarun Kumar, S Sanjana, and Kriti Bhalla. Curated dataset for COVID-19 posterior-anterior chest radiography images (x-rays). Mendeley Data V4, 1:1, 2022.
[27]Ercan Avs¸ar. Effects of image preprocessing on the performance of convolutional neural networks for pneumonia detection. In 2021 International Conference on INnovations in Intelligent SysTems and Applications (INISTA), pages 1–5. IEEE, 2021.
[28]Lalit Narayan and Virendra Vishwakarma. A Review on Different Image Enhancement Techniques, pages 143–153. 08 2023.
[29]Maider Abad, Jordi Casas-Roma, and Ferran Prados. Reducing the Learning Domain by Using Image Processing to Diagnose COVID-19 from X-Ray Image. 10 2022.
[30]Garima Yadav, Saurabh Maheshwari, and Anjali Agarwal. Contrast limited adaptive histogram equalization based enhancement for real time video system. In 2014 international conference on advances in computing, communications and informatics (ICACCI), pages 2392–2397. IEEE, 2014.
[31]Carl Sabottke and Bradley Spieler. The effect of image resolution on deep learning in radiography. Radiology: Artificial Intelligence, 2:e190015, 01 2020.
[32]Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1), 2017.
[33]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[34]Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[35]Yuan Yang, Lin Zhang, Mingyu Du, Jingyu Bo, Haolei Liu, Lei Ren, Xiaohe Li, and M. Jamal Deen. A comparative analysis of eleven neural networks architectures for small datasets of lung images of COVID-19 patients toward improved clinical decisions. Computers in Biology and Medicine, 139:104887, 2021.
[36]Ramsey M. Wehbe, Jiayue Sheng, Shinjan Dutta, Siyuan Chai, Amil Dravid, Semih Barutcu, Yunan Wu, Donald R. Cantrell, Nicholas Xiao, Bradley D. Allen, Gregory A. MacNealy, Hatice Savas, Rishi Agrawal, Nishant Parekh, and Aggelos K. Katsaggelos. DeepCOVID-XR: An artificial intelligence algorithm to detect COVID-19 on chest radiographs trained and tested on a large u.s. clinical data set. Radiology, 299(1):E167–E176, 2021. PMID: 33231531.
[37]Ashley Gillman, Febrio Lunardo, Joseph Prinable, Gregg Belous, Aaron Nicolson, Min Hang, Andrew Terhorst, and Jason Dowling. Automated COVID-19 diagnosis and prognosis with medical imaging and who is publishing: a systematic review. Physical and Engineering Sciences in Medicine, 45, 12 2021.
[38]Chandana Panati, Simon Wagner, and Stefan Brueggenwirth. Feature relevance evaluation using grad-cam, lime and shap for deep learning sar data classification. pages 457–462, 09 2022.
[39]Qingyu Zhao, Zixuan Liu, Ehsan Adeli, and Kilian M. Pohl. Longitudinal self-supervised learning. Medical Image Analysis, 71:102051, 2021.