  Aspect based sentiment analysis aims to identify the sentimental tendency
towards a given aspect in text. Fine-tuning of pretrained BERT performs
excellent on this task and achieves state-of-the-art performances. Existing
BERT-based works only utilize the last output layer of BERT and ignore the
semantic knowledge in the intermediate layers. This paper explores the
potential of utilizing BERT intermediate layers to enhance the performance of
fine-tuning of BERT. To the best of our knowledge, no existing work has been
done on this research. To show the generality, we also apply this approach to a
natural language inference task. Experimental results demonstrate the
effectiveness and generality of the proposed approach.
