  Handwritten character recognition (HCR) remains a challenging pattern
recognition problem despite decades of research, and lacks research on script
independent recognition techniques. {\color{black}This is mainly because of
similar character structures, different handwriting styles, diverse scripts,
handcrafted feature extraction techniques, unavailability of data and code, and
the development of script-specific deep learning techniques. To address these
limitations, we have proposed a script independent deep learning network for
HCR research, called HCR-Net, that sets a new research direction for the field.
HCR-Net is based on a novel transfer learning approach for HCR, which
\textit{partly utilizes} feature extraction layers of a pre-trained network.}
Due to transfer learning and image augmentation, HCR-Net provides faster and
computationally efficient training, better performance and generalizations, and
can work with small datasets. HCR-Net is extensively evaluated on 40 publicly
available datasets of Bangla, Punjabi, Hindi, English, Swedish, Urdu, Farsi,
Tibetan, Kannada, Malayalam, Telugu, Marathi, Nepali and Arabic languages, and
established 26 new benchmark results while performed close to the best results
in the rest cases. HCR-Net showed performance improvements up to 11\% against
the existing results and achieved a fast convergence rate showing up to 99\% of
final performance in the very first epoch. HCR-Net significantly outperformed
the state-of-the-art transfer learning techniques and also reduced the number
of trainable parameters by 34\% as compared with the corresponding pre-trained
network. To facilitate reproducibility and further advancements of HCR
research, the complete code is publicly released at
\url{https://github.com/jmdvinodjmd/HCR-Net}.
