  Rapid growth of the biomedical literature has led to many advances in the
biomedical text mining field. Among the vast amount of information, biomedical
article abstracts are the easily accessible sources. However, the number of the
structured abstracts, describing the rhetorical sections with one of
Background, Objective, Method, Result and Conclusion categories is still not
considerable. Exploration of valuable information in the biomedical abstracts
can be expedited with the improvements in the sequential sentence
classification task. Deep learning based models has great performance/potential
in achieving significant results in this task. However, they can often be
overly complex and overfit to specific data. In this project, we study a
state-of-the-art deep learning model, which we called SSN-4 model here. We
investigate different components of the SSN-4 model to study the trade-off
between the performance and complexity. We explore how well this model
generalizes to a new data set beyond Randomized Controlled Trials (RCT)
dataset. We address the question that whether word embeddings can be adjusted
to the task to improve the performance. Furthermore, we develop a second model
that addresses the confusion pairs in the first model. Results show that SSN-4
model does not appear to generalize well beyond RCT dataset.
