  In this paper we address a task of relation mention extraction from noisy
data: extracting representative phrases for a particular relation from noisy
sentences that are collected via distant supervision. Despite its significance
and value in many downstream applications, this task is less studied on noisy
data. The major challenges exists in 1) the lack of annotation on mention
phrases, and more severely, 2) handling noisy sentences which do not express a
relation at all. To address the two challenges, we formulate the task as a
semi-Markov decision process and propose a novel hierarchical reinforcement
learning model. Our model consists of a top-level sentence selector to remove
noisy sentences, a low-level mention extractor to extract relation mentions,
and a reward estimator to provide signals to guide data denoising and mention
extraction without explicit annotations. Experimental results show that our
model is effective to extract relation mentions from noisy data.
