  In relation extraction with distant supervision, noisy labels make it
difficult to train quality models. Previous neural models addressed this
problem using an attention mechanism that attends to sentences that are likely
to express the relations. We improve such models by combining the distant
supervision data with an additional directly-supervised data, which we use as
supervision for the attention weights. We find that joint training on both
types of supervision leads to a better model because it improves the model's
ability to identify noisy sentences. In addition, we find that sigmoidal
attention weights with max pooling achieves better performance over the
commonly used weighted average attention in this setup. Our proposed method
achieves a new state-of-the-art result on the widely used FB-NYT dataset.
