  In brain signal processing, deep learning (DL) models have become commonly
used. However, the performance gain from using end-to-end DL models compared to
conventional ML approaches is usually significant but moderate, typically at
the cost of increased computational load and deteriorated explainability. The
core idea behind deep learning approaches is scaling the performance with
bigger datasets. However, brain signals are temporal data with a low
signal-to-noise ratio, uncertain labels, and nonstationary data in time. Those
factors may influence the training process and slow down the models'
performance improvement. These factors' influence may differ for end-to-end DL
model and one using hand-crafted features. As not studied before, this paper
compares models that use raw ECoG signal and time-frequency features for BCI
motor imagery decoding. We investigate whether the current dataset size is a
stronger limitation for any models. Finally, obtained filters were compared to
identify differences between hand-crafted features and optimized with
backpropagation. To compare the effectiveness of both strategies, we used a
multilayer perceptron and a mix of convolutional and LSTM layers that were
already proved effective in this task. The analysis was performed on the
long-term clinical trial database (almost 600 minutes of recordings) of a
tetraplegic patient executing motor imagery tasks for 3D hand translation. For
a given dataset, the results showed that end-to-end training might not be
significantly better than the hand-crafted features-based model. The
performance gap is reduced with bigger datasets, but considering the increased
computational load, end-to-end training may not be profitable for this
application.
