Handwritten character recognition is a crucial challenge in artificial intelligence and computer vision, particularly for complex scripts like Devanagari. Devanagari is widely used in Hindi, Marathi, and Nepali and consists of intricate multilevel compound characters, ligatures, and highly variable handwriting styles. Despite advances in optical character recognition (OCR) technology, accurately recognizing handwritten Devanagari characters remains difficult. This study compares various deep learning models, including Convolutional Neural Networks (CNN), CNN-SVM, Long Short-Term Memory (LSTM), EfficientNet, and a newly proposed attention-based CNN model. Extensive experiments were conducted on a diverse dataset containing simple and compound handwritten Devanagari characters. The proposed attention-based CNN model outperforms traditional methods, achieving a recognition accuracy of 96.5%, significantly higher than CNN (88.0%), CNN-SVM (88.5%), LSTM (92.0%), and EfficientNet (93.0%). The study employs advanced data augmentation techniques to enhance model robustness, making it adaptable to various handwriting styles. The attention mechanism in the proposed CNN model allows for improved feature extraction, leading to higher recognition accuracy and efficiency. This research contributes to developing robust OCR systems for the Devanagari script, enabling improved digitisation and preservation of Indian languages. The proposed approach can be extended to other complex scripts like Bengali and Tamil, further advancing multilingual OCR technologies, is now exploratory and has not been experimentally verified in this study. Future work can address the need for thorough cross-script evaluation and transfer learning studies to verify the adaptability of the attention-based CNN architecture, despite its inherent script-agnostic nature. The findings of this study hold significant implications for text digitization, historical document preservation, and automated language processing applications.