  With the development of deep neural language models, great progress has been
made in information extraction recently. However, deep learning models often
overfit on noisy data points, leading to poor performance. In this work, we
examine the role of information entropy in the overfitting process and draw a
key insight that overfitting is a process of overconfidence and entropy
decreasing. Motivated by such properties, we propose a simple yet effective
co-regularization joint-training framework TIER-A, Aggregation Joint-training
Framework with Temperature Calibration and Information Entropy Regularization.
Our framework consists of several neural models with identical structures.
These models are jointly trained and we avoid overfitting by introducing
temperature and information entropy regularization. Extensive experiments on
two widely-used but noisy datasets, TACRED and CoNLL03, demonstrate the
correctness of our assumption and the effectiveness of our framework.
