IJCNIS Vol. 18, No. 2, 8 Apr. 2026
Cover page and Table of Contents: PDF (size: 1360KB)
PDF (1360KB), PP.61-77
Views: 0 Downloads: 0
Malware Analysis, Machine Learning, Deep Learning, Malware Detection, Static Malware Analysis, Dynamic Malware Analysis
With the rapid increase in malware threats, robust classification methods have become essential to protect digital environments. This study conducts a comparative analysis of machine learning and deep learning methods for malware detection. A variety of models are used from both machine learning and deep learning paradigms to determine their effectiveness in distinguishing malware. To further refine the models, several feature selection techniques are applied to reduce the dimensionality of the data and enhance performance. Performance metrics, including accuracy, precision, recall, and F1-score is used to evaluate each model. The findings indicate that while deep learning approaches generally provide higher detection accuracy, feature selection methods contribute significantly to improving machine learning models in terms of performance and computational efficiency. This analysis offers valuable insights into the balance between model complexity and effectiveness, providing practical recommendations for implementing malware classification systems in real-world applications.
Nayankumar M. Mali, Narendrasinh C. Chauhan, "Advances in Malware Detection using Machine Learning and Deep Learning: A Comprehensive Comparative Analysis", International Journal of Computer Network and Information Security(IJCNIS), Vol.18, No.2, pp.61-77, 2026. DOI:10.5815/ijcnis.2026.02.04
[1]Edward Raff and Charles Nicholas. A survey of machine learning methods and challenges for windows malware classification. Cornell University, 01 2020.
[2]Ahmed Bensaoud, Jugal Kalita, and Mahmoud Bensaoud. A survey of malware detection using deep learning. Elsevier BV, 16:100546–100546, 03 2024.
[3]Md. Alamgir Hossain and Md. Saiful Islam. Enhanced detection of obfuscated malware in memory dumps: a machine learning approach for advanced cybersecurity. Springer Nature, 7(1), 01 2024.
[4]Ban Mohammed Khammas, Alireza Monemi, Joseph Stephen Bassi, Ismahani Ismail, Sulaiman Mohd Nor, and M. N. Marsono. Feature selection and machine learning classification for malware detection. Muhammadiyah University of Jakarta, 77(1), 10 2015.
[5]Tony Quertier, Benjamin Marais, Stephane Morucci, and Bertrand Fournel. Merlin – malware evasion with rein-´ forcement learning. Cornell University, 01 2022.
[6]Chun Yang, Jinghui Xu, Shuangshuang Liang, Yanna Wu, Yu Wen, Boyang Zhang, and Dan Meng. Deepmal: maliciousness-preserving adversarial instruction learning against static malware detection. Springer Nature, 4(1), 05 2021.
[7]Khatoon Mohammed. Harnessing the speed and accuracy of machine learning to advance cybersecurity. Cornell University, 01 2023.
[8]Sajedul Talukder and Zahidur Talukder. A survey on malware detection and analysis tools. 12(2):37–57, 03 2020.
[9]Neil Balram, George Hsieh, and McFall Christian. Static malware analysis using machine learning algorithms on apt1 dataset with string and pe header features. 12 2019.
[10]Abir Laouadi, Djamel Eddine Menacer, and Karima Benatchba. A machine learning approach for malware detection based on image conversion, 01 2024.
[11]Charles Nicholas and Robert N. Brandon. Document engineering issues in malware analysis. 09 2016.
[12]Marcus Botacin, Paulo L´ıcio Geus, and Andre Gr´ egio. Enhancing branch monitoring for security purposes: From´ control flow integrity to malware analysis and debugging, 01 2018.
[13]Chin-Wei Tien, Shang-Wen Chen, Tao Ban, and Sy-Yen Kuo. Machine learning framework to analyze iot malware using elf and opcode features. Association for Computing Machinery, 1(1):1–19, 03 2020.
[14]Richard C. Linger, Stacy Prowell, and Kirk Sayre. Computing the behavior of malicious code with function extraction technology. 04 2009.
[15]Mi-Jung Choi, Jiwon Bang, Jongwook Kim, Hajin Kim, and Yang-Sae Moon. All-in-one framework for detection, unpacking, and verification for malware analysis. Hindawi Publishing Corporation, 2019:1–16, 10 2019.
[16]Selvy Megira, A R Pangesti, and Ferry Wahyu Wibowo. Malware analysis and detection using reverse engineering technique. IOP Publishing, 1140:012042–012042, 12 2018.
[17]Mohammadreza Ebrahimi, Ning Zhang, James Lee Hu, Muhammad Taqi Raza, and Hsinchun Chen. Binary blackbox evasion attacks against deep learning-based static malware detectors with adversarial byte-level language model, 01 2020.
[18]Suarez de Tangil Rotaeche and Guillermo Nicol´ as. Mining structural and behavioral patterns in smart malware, 11´ 2016.
[19]Pascal Maniriho, Abdun Naser Mahmood, and Mohammad Jabed Morshed Chowdhury. Api-maldetect: Automated malware detection framework for windows based on api calls and deep learning techniques. Elsevier BV, 218:103704–103704, 09 2023.
[20]Ayorinde Henry Omopintemi, Ibrahim Ghafir, Shadi Eltanani, Sohag Kabir, and Moemedi Lefoane. Machine learning for malware detection in network traffic, 12 2023.
[21]Edward Raff, William Fleshman, Richard Zak, Hyrum S. Anderson, Bobby Filar, and Mark McLean. Classifying sequences of extreme length with constant memory applied to malware detection. Association for the Advancement of Artificial Intelligence, 35(11):9386–9394, 05 2021.
[22]Christian Catalano, Andrea Chezzi, Mario Angelelli, and Franco Tommasi. Deceiving ai-based malware detection through polymorphic attacks. Elsevier BV, 143:103751–103751, 12 2022.
[23]Jashanpreet Singh Sraw and Keshav Kumar. Using static and dynamic malware features to perform malware ascription, 01 2021.
[24]Adam Wolsey. The state-of-the-art in ai-based malware detection techniques: A review, 01 2022.
[25]Mustafa Irshad, Haider Al-Khateeb, Ali Mansour, Moses Ashawa, and Muhammad Hamisu. Effective methods to detect metamorphic malware: a systematic review, 01 2018.
[26]LI Xu-fang, Peter K. K. Loh, and Freddy Susanto Tan. Mechanisms of polymorphic and metamorphic viruses. 09 2011.
[27]Anusha Damodaran, Fabio Di Troia, Corrado Aaron Visaggio, Thomas H. Austin, and Mark Stamp. A comparison of static, dynamic, and hybrid analysis for malware detection. Springer Science+Business Media, 13(1):1–12, 12 2015.
[28]Shabnam Aboughadareh, Christoph Csallner, and Mehdi Azarmi. Mixed-mode malware and its analysis. 12 2014.
[29]G. Ganesh Sundarkumar, Vadlamani Ravi, Ifeoma Nwogu, and Venu Govindaraju. Malware detection via api calls, topic models and machine learning. 08 2015.
[30]Chen Li and Junjun Zheng. Api call-based malware classification using recurrent neural networks, 05 2021.
[31]Muhammad Shoaib Akhtar and Tao Feng. Malware analysis and detection using machine learning algorithms. Multidisciplinary Digital Publishing Institute, 14(11):2304–2304, 11 2022.
[32]Zhigang Zhang, Chaowen Chang, Peisheng Han, and Hongtao Zhang. Packed malware variants detection using deep belief networks. EDP Sciences, 309:02002–02002, 01 2020.
[33]Matthew G. Schultz, Eleazar Eskin, F. Zadok, and Sal Stolfo. Data mining methods for detection of new malicious executables. 11 2002.
[34]Igor Santos, Felix Brezo, Javier Nieves, Yoseba K. Penya, Borja Sanz, Carlos Laorden, and Pablo G. Bringas. Idea:´ Opcode-sequence-based malware detection. Springer Science+Business Media, pages 35–43, 01 2010.
[35]Shifu Hou, Aaron Saas, Lifei Chen, and Yanfang Ye. Deep4maldroid: A deep learning framework for android malware detection based on linux kernel system call graphs. 10 2016.
[36]Hyrum S. Anderson and Phil Roth. Ember: An open dataset for training static pe malware machine learning models, 01 2018.
[37]Zhiqiang Wang, Qian Liu, and Yaping Chi. Review of android malware detection based on deep learning. Institute of Electrical and Electronics Engineers, 8:181102–181126, 01 2020.
[38]Youness Mourtaji, Mohammed Bouhorma, and Daniyal Alghazzawi. Intelligent framework for malware detection with convolutional neural network. 03 2019.
[39]Luca Demetrio, Scott E. Coull, Battista Biggio, Giovanni Lagorio, Alessandro Armando, and Fabio Roli. Adversarial exemples. Association for Computing Machinery, 24(4):1–31, 09 2021.
[40]Kamran Shaukat, Suhuai Luo, and Vijay Varadharajan. A novel machine learning approach for detecting first-timeappeared malware. Elsevier BV, 131:107801–107801, 05 2024.
[41]Gulsade Kale, Erkan Bostancı, and Fatih V. C¸elebi. Evolutionary feature selection for machine learning based malware classification. Elsevier BV, 56:101762–101762, 08 2024.
[42]Ethan M. Rudd, David Krisiloff, Scott E. Coull, Daniel Olszewski, Edward Raff, and James Holt. Efficient malware analysis using metric embeddings. Association for Computing Machinery, 5(1):1–20, 03 2024.
[43]Shamika Ganesan, Vinayakumar Ravi, Moez Krichen, V. Sowmya, Roobaea Alroobaea, and K. P. Soman. Robust malware detection using residual attention network. 01 2021.
[44]Konrad Rieck, Philipp Trinius, Carsten Willems, and Thorsten Holz. Automatic analysis of malware behavior using machine learning. IOS Press, 19(4):639–668, 06 2011.
[45]Bojan Kolosnjaji, Apostolis Zarras, George Webster, and Claudia Eckert. Deep learning for classification of malware system call sequences.
[46]Joshua Saxe and Konstantin Berlin. Deep neural network based malware detection using two dimensional binary program features. Cornell University, 01 2015.
[47]Edward Raff, Jon Barker, Jared Sylvester, Robert N. Brandon, Bryan Catanzaro, and Charles Nicholas. Malware detection by eating a whole exe, 01 2017.
[48]Abdullah Al-Dujaili, Alex Huang, Erik Hemberg, and Una-May O’Reilly. Adversarial deep learning for robust detection of binary encoded malware. 05 2018.
[49]Omid E. David and Nathan S. Netanyahu. Deepsign: Deep learning for automatic malware signature generation and classification. 07 2015.
[50]Kamalakanta Sethi, Rahul Kumar, Lingaraj Sethi, Padmalochan Bera, and Prashanta Kumar Patra. A novel machine learning based malware detection and classification framework. 06 2019.
[51]Ismahani Ismail, Sulaiman Mohd Nor, and M. N. Marsono. Stateless malware packet detection by incorporating naive bayes with known malware signatures. Hindawi Publishing Corporation, 2014:1–8, 01 2014.
[52]Sudesh Kumar, Shersingh, Siddhant kumar, and Kinshuk Verma. Malware classification using machine learning models. Elsevier BV, 235:1419–1428, 01 2024.
[53]Santosh K. Smmarwar, Govind P. Gupta, and Sanjay Kumar. Android malware detection and identification frameworks by leveraging the machine and deep learning techniques: A comprehensive review, 03 2024.
[54]Jemal Abawajy, Abdulbasit A. Darem, and Asma A. Alhashmi. Feature subset selection for malware detection in smart iot platforms. Multidisciplinary Digital Publishing Institute, 21(4):1374–1374, 02 2021.
[55]Thiago Guzella and Walmir M. Caminhas. A review of machine learning approaches to spam filtering, 02 2009.
Alessandro Panconesi, Marian, Will Cukierski, and WWW BIG Cup Committee. Microsoft malware classification challenge (big 2015). https://kaggle.com/competitions/malware-classification, 2015. Kaggle.