  Text segmentation holds paramount importance in the field of Natural Language
Processing (NLP). It plays an important role in several NLP downstream tasks
like information retrieval and document summarization. In this work, we propose
a new solution, namely TocBERT, for segmenting texts using bidirectional
transformers. TocBERT represents a supervised solution trained on the detection
of titles and sub-titles from their semantic representations. This task was
formulated as a named entity recognition (NER) problem. The solution has been
applied on a medical text segmentation use-case where the Bio-ClinicalBERT
model is fine-tuned to segment discharge summaries of the MIMIC-III dataset.
The performance of TocBERT has been evaluated on a human-labeled ground truth
corpus of 250 notes. It achieved an F1-score of 84.6% when evaluated on a
linear text segmentation problem and 72.8% on a hierarchical text segmentation
problem. It outperformed a carefully designed rule-based solution, particularly
in distinguishing titles from subtitles.
