  The popularity of Windows attracts the attention of hackers/cyber-attackers,
making Windows devices the primary target of malware attacks in recent years.
Several sophisticated malware variants and anti-detection methods have been
significantly enhanced and as a result, traditional malware detection
techniques have become less effective. This work presents MalBehavD-V1, a new
behavioural dataset of Windows Application Programming Interface (API) calls
extracted from benign and malware executable files using the dynamic analysis
approach. In addition, we present MalDetConV, a new automated behaviour-based
framework for detecting both existing and zero-day malware attacks. MalDetConv
uses a text processing-based encoder to transform features of API calls into a
suitable format supported by deep learning models. It then uses a hybrid of
convolutional neural network (CNN) and bidirectional gated recurrent unit
(CNN-BiGRU) automatic feature extractor to select high-level features of the
API Calls which are then fed to a fully connected neural network module for
malware classification. MalDetConv also uses an explainable component that
reveals features that contributed to the final classification outcome, helping
the decision-making process for security analysts. The performance of the
proposed framework is evaluated using our MalBehavD-V1 dataset and other
benchmark datasets. The detection results demonstrate the effectiveness of
MalDetConv over the state-of-the-art techniques with detection accuracy of
96.10%, 95.73%, 98.18%, and 99.93% achieved while detecting unseen malware from
MalBehavD-V1, Allan and John, Brazilian, and Ki-D datasets, respectively. The
experimental results show that MalDetConv is highly accurate in detecting both
known and zero-day malware attacks on Windows devices.
