Arrhythmias are irregularities in heartbeats and hence accurate classification of arrhythmia has great importance for administering patients to the right cardiac care. This paper presents a five-class arrhythmia classification framework using Encoded Transformer (ET) based Convolutional Neural Network and Long Short-Term Memory (CNN-ET-LSTM) hybrid model to ECG signal. The dataset used in this research is the widely used MIT-BIH arrhythmia database that has five distinct types of arrhythmia: non-ectopic beats (N), ventricular ectopic beats (V), supraventricular ectopic beats (S), fusion beats (F), and unknown beats (Q). The class imbalance problem is dealt by utilizing Synthetic Minority Oversampling Technique (SMOTE) that has an impact for bettering the performance especially on minority classes. In the proposed CNN-ET-LSTM model, the CNN is used as a feature extractor and the long range dependencies in the ECG waveform are captured by the encoded transformer module. The LSTM layers are used to processes features sequentially to feed them to the fully connected layers for classification.  Experimental results showed that the proposed system achieved an accuracy of 97.52%, precision of 97.80%, recall of 97.52% and F1-score of 97.62% with raw blind test data. The performance of our model is also compared to other existing methods that use the same dataset and found useful for clinical applications.