  Social media is becoming an increasingly important source of information to
complement traditional pharmacovigilance methods. In order to identify signals
of potential adverse drug reactions, it is necessary to first identify medical
concepts in the social media text. Most of the existing studies use
dictionary-based methods which are not evaluated independently from the overall
signal detection task.
  We compare different approaches to automatically identify and normalise
medical concepts in consumer reviews in medical forums. Specifically, we
implement several dictionary-based methods popular in the relevant literature,
as well as a method we suggest based on a state-of-the-art machine learning
method for entity recognition. MetaMap, a popular biomedical concept extraction
tool, is used as a baseline. Our evaluations were performed in a controlled
setting on a common corpus which is a collection of medical forum posts
annotated with concepts and linked to controlled vocabularies such as MedDRA
and SNOMED CT.
  To our knowledge, our study is the first to systematically examine the effect
of popular concept extraction methods in the area of signal detection for
adverse reactions. We show that the choice of algorithm or controlled
vocabulary has a significant impact on concept extraction, which will impact
the overall signal detection process. We also show that our proposed machine
learning approach significantly outperforms all the other methods in
identification of both adverse reactions and drugs, even when trained with a
relatively small set of annotated text.
