  Undoubtedly that the Bidirectional Encoder representations from Transformers
is the most powerful technique in making Natural Language Processing tasks such
as Named Entity Recognition, Question & Answers or Sentiment Analysis, however,
the use of traditional techniques remains a major potential for the improvement
of recent models, in particular word tokenization techniques and embeddings,
but also the improvement of neural network architectures which are now the core
of each architecture. recent. In this paper, we conduct a comparative study
between Fine-Tuning the Bidirectional Encoder Representations from Transformers
and a method of concatenating two embeddings to boost the performance of a
stacked Bidirectional Long Short-Term Memory-Bidirectional Gated Recurrent
Units model; these two approaches are applied in the context of sentiment
analysis of shopping places in Morocco. A search for the best learning rate was
made at the level of the two approaches, and a comparison of the best
optimizers was made for each sentence embedding combination with regard to the
second approach.
