  Prediction of binding sites for transcription factors is important to
understand how they regulate gene expression and how this regulation can be
modulated for therapeutic purposes. Although in the past few years there are
significant works addressing this issue, there is still space for improvement.
In this regard, a transformer based capsule network viz. DNABERT-Cap is
proposed in this work to predict transcription factor binding sites mining
ChIP-seq datasets. DNABERT-Cap is a bidirectional encoder pre-trained with
large number of genomic DNA sequences, empowered with a capsule layer
responsible for the final prediction. The proposed model builds a predictor for
transcription factor binding sites using the joint optimisation of features
encompassing both bidirectional encoder and capsule layer, along with
convolutional and bidirectional long-short term memory layers. To evaluate the
efficiency of the proposed approach, we use a benchmark ChIP-seq datasets of
five cell lines viz. A549, GM12878, Hep-G2, H1-hESC and Hela, available in the
ENCODE repository. The results show that the average area under the receiver
operating characteristic curve score exceeds 0.91 for all such five cell lines.
DNABERT-Cap is also compared with existing state-of-the-art deep learning based
predictors viz. DeepARC, DeepTF, CNN-Zeng and DeepBind, and is seen to
outperform them.
