  The number of Language Models (LMs) dedicated to processing scientific text
is on the rise. Keeping pace with the rapid growth of scientific LMs (SciLMs)
has become a daunting task for researchers. To date, no comprehensive surveys
on SciLMs have been undertaken, leaving this issue unaddressed. Given the
constant stream of new SciLMs, appraising the state-of-the-art and how they
compare to each other remain largely unknown. This work fills that gap and
provides a comprehensive review of SciLMs, including an extensive analysis of
their effectiveness across different domains, tasks and datasets, and a
discussion on the challenges that lie ahead.
