  Handwritten character recognition is a hot topic for research nowadays. If we
can convert a handwritten piece of paper into a text-searchable document using
the Optical Character Recognition (OCR) technique, we can easily understand the
content and do not need to read the handwritten document. OCR in the English
language is very common, but in the Bengali language, it is very hard to find a
good quality OCR application. If we can merge machine learning and deep
learning with OCR, it could be a huge contribution to this field. Various
researchers have proposed a number of strategies for recognizing Bengali
handwritten characters. A lot of ML algorithms and deep neural networks were
used in their work, but the explanations of their models are not available. In
our work, we have used various machine learning algorithms and CNN to recognize
handwritten Bengali digits. We have got acceptable accuracy from some ML
models, and CNN has given us great testing accuracy. Grad-CAM was used as an
XAI method on our CNN model, which gave us insights into the model and helped
us detect the origin of interest for recognizing a digit from an image.
